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Summary 

This report consists of a survey of the state of the art in uncertainty-based design together with 
recommendations for a Base research activity in this area for the NASA Langley Research Center. In 
particular, it focuses on the needs and opportunities for computational and experimental methods that 
provide accurate, efficient solutions to nondeterministic multidisciplinary aerospace vehicle design 
problems. We use the term uncertainty-based design to describe this type of design method. The two 
major classes of uncertainty-based design problems are robust design problems and reliability-based 
design problems. A robust design problem seeks a design that is relatively insensitive to small changes in 
the uncertain quantities. A reliability-based design seeks a design that has a probability of failure that is 
less than some acceptable (invariably small) value. 

Traditional design procedures for aerospace vehicle structures are based on combinations of factors of 
safety and knockdown factors. The aerodynamic design procedures used by the industry are exclusively 
deterministic. There has been considerable work on “robust controls, 1 ’ but this work has been limited to 
using norm bounds on the uncertain variables. Reliability-based design methods have been used within 
civil engineering for several decades and in aircraft engine design for about a decade. Applications to the 
structural design of airframes are only now starting to emerge. Only academic studies of reliability-based 
design methods within the aerodynamics and controls disciplines are known to the authors. 

To use uncertainty-based design methods, the various uncertainties associated with the design problem 
must be characterized and managed, and these characterizations must be exploited. In the context of 
computational modeling and simulation, two complementary categorizations of uncertainties are useful. 
One categorization distinguishes between parameter uncertainties and model form uncertainties. 
Parameter uncertainties are those uncertainties associated either with the input data (boundary conditions 
or initial conditions) to a computational process or with basic parameters that define a given 
computational process, such as the coefficients of phenomenological models. Model form uncertainties 
are uncertainties associated with model validity, i.e., whether the nominal mathematical model adequately 
captures the physics of the problem. Systematic procedures for characterizing and managing uncertainties 
in experimental activities include design of experiment methods and statistical process control techniques. 
The former focuses more on characterizing the uncertainties and the latter more on managing them. 

Parameter uncertainties are typically specified in terms of probability density functions, membership 
functions, or interval bounds. Model form uncertainties are very difficult to characterize. Generic 
techniques are available for assessing the effects of uncertainties on discipline and system performance 
predictions, and some optimization methods can account for uncertainties. However, better and less 
resource-intensive methods are needed for both uncertainty propagation and optimization under 
uncertainty. Certainly, the deployment of existing and new techniques within the aerodynamic, controls, 
structures, and systems analysis disciplines for applications to aerospace vehicles is critically needed. 

The principal barriers to the adoption of uncertainty-based design methods for aerospace vehicles are 
as follows: 

Bl. Industry feels comfortable with traditional design methods 

B2. Few demonstrations of the benefits of uncertainty-based design methods are available. 

B3. Current uncertainty-based design methods are more complex and much more 
computationally expensive than deterministic methods. 
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B4. Characterization of structural imperfections and uncertainties necessary to facilitate accurate 
analysis and design of the structure is time-consuming and is highly dependent on structural 
configuration, material system, and manufacturing processes. 

B5. There is a dearth of statistical process control activity in aerodynamics. 

B6. Effective approaches for characterizing model form error are lacking. 

B7. There are no dependable approaches to uncertainty quantification for nonlinear problems. 

B8. Characterization of uncertainties for use in control is inadequate. 

B9. Methods for mapping probabilistic parameter uncertainties into norm-bounded uncertainties 
do not exist. 

BIO. Existing probabilistic analysis tools are not well suited to handle the time and frequency 
domain response quantities that are typically used in the analysis of closed-loop dynamical 
systems. 

B 1 1 . No methods are available for optimization under nonprobabilistic uncertainties. 

B 1 2. Current methods for optimization under uncertainty are too expensive for use with high- 
fidelity analysis tools in many disciplines. 

B13. Extending uncertainty analysis and optimization to applications involving multiple 
disciplines compounds the complexity and cost. 

B 14. Researchers and analysts lack training in statistical methods and probabilistic assessment. 

The principal benefits of uncertainty-based design are 

PI. Confidence in analysis tools will increase. 

P2. Design cycle time, cost, and risk will be reduced. 

P3. System performance will increase while ensuring that reliability requirements are met. 

P4. Designs will be more robust. 

P5. The methodology can assess systems at off-nominal conditions. 

P6. Use of composite structures will increase. 

The proposed role for NASA Langley Research Center in uncertainty-based design is: 

Evaluate and improve methods for management of uncertainty with applications to 
multidisciplinary aerospace vehicle design by developing and validating strategies, algorithms, 
tools and data to 

characterize and manage the uncertainties from the individual aerospace vehicle design 
disciplines, especially aerodynamics, structures, and controls, based on the best available 
experimental and computational results; 

characterize the norm and distribution of the resulting uncertainties in system metrics; 
and 

account for uncertainties in the design of aerospace vehicles at the conceptual through the 
detailed design stages. 

Detailed lists of uncertainty-based design technology needs for the structures, aerodynamics, controls, and 
systems analysis disciplines are found in section 4. 
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1. Introduction 


This white paper focuses on the needs and opportunities for computational and experimental methods 
that provide accurate, efficient solutions to problems of multidisciplinary aerospace vehicle design in the 
presence of uncertainties. These methods are a subset of what are sometimes referred to as 
nondeterministic approaches. The essential distinction is between the formulations of the design problem 
and the methods used for its solution. A nondeterministic problem formulation is one in which some 
essential components — the problem statement (e.g., uncertainty of the outer mold line due to 
manufacturing variability), experimental data (e.g., measurement uncertainty), or computational solutions 
(e.g., discretization error) — are treated as nondeterministic. The uncertain aspects may be expressed in a 
number of ways, for example by interval bounds or by probability density functions. Analysis methods 
that employ stochastic approximations, such as Monte Carlo approximation of integrals, are only of 
interest here to the extent that they are brought to bear on a genuinely nondeterministic problem 
formulation. Likewise, random search techniques, such as genetic algorithms and simulated annealing, are 
not intrinsically of interest in the present context. We use the term uncertainty-based design to describe 
those design problems that have a nondeterministic formulation. 

Impact of event 



Everyday fluctuations Extreme events 

Frequency of event 


Figure 1 . Uncertainty-based design domains (from Huyse 2001 ). 

The two major classes of uncertainty-based design problems are robust design problems and 
reliability-based design problems. A robust design problem is one in which a design is sought that is 
relatively insensitive to small changes in the uncertain quantities. A reliability-based design problem is 
one in which a design is sought that has a probability of failure that is less than some acceptable 
(invariably small) value. The same abstract mathematical formulation can be used to describe both robust 
design and reliability-based design. However, their domains of applicability are rather different. 

Figure I illustrates these domains. The two major factors are the frequency of the event and the impact 
of the event. No system is viable if everyday fluctuations can lead to catastrophe. Instead, one would like 
the system to be designed such that the performance is insensitive, i.e., robust, to everyday fluctuations. 
On the other hand, one would like to ensure that the events that lead to catastrophe are extremely unlikely. 
This is the domain of reliability-based design. In both cases, the design risk is a combination of the 
likelihood of an undesired event and the consequences of that event. An example of risk in the robust 


3 



design context is the likelihood that the aircraft design will fail to meet the aerodynamic performance 
targets and will consequently lose sales and perhaps even go bankrupt. An example of risk in the 
reliability-based design context is the probability that a critical structural component will fail, which 
could lead to the loss of the vehicle or spacecraft, payload, and passengers, and to potential lawsuits. 

Probability 

density 



Figure 2. Reliability versus robustness in terms of the probability density function. 

As figure 2 illustrates, robust design is concerned with the event distribution near the mean of the 
probability density function, whereas reliability-based design is concerned with the event distribution in 
the tails of the probability density function. Obviously, it is much more difficult to accurately characterize 
the tail of a distribution than the center of the distribution. An additional consideration in distinguishing 
between robustness and reliability is that the mathematical techniques used for solving robust design 
problems are considerably different from those used for solving reliability-based design problems. The 
mathematical methods for robust design procedures are less well developed than those for reliability- 
based design procedures, and this work is still largely confined to academic studies. Certainly, the 
aerodynamic design procedures in use in industry are exclusively deterministic. (Recall that we are 
excluding the use of random search methods to solve a deterministic problem.) There has been 
considerable work on “robust controls,” but this work has been limited to using norm-bounded 
descriptions of uncertain variables. Although the robust design principles of Taguchi (1987) are used in 
aerospace engineering, these are not necessarily the best or even appropriate methods for many robust 
design problems. 

Traditional design procedures for aerospace vehicle structures are based on combinations of factors of 
safety and knockdown factors, as illustrated in figure 3. Factors of safety are numbers greater than 1.0 that 
are applied to the loads. Knockdown factors are numbers less than 1.0 that are applied to the strengths. 
Both factors are intended to account for uncertainties. They have proven useful during nearly six decades 
of design for conventional metal airframes. 
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Factor of safety Knockdown factor 



Figure 3. Factor of safety approach. 

Traditional design procedures have several shortcomings. First, these procedures may be difficult to 
apply to aerospace vehicles that have unconventional configurations and that use new material systems. 
Second, measures of reliability and robustness are not provided in the design process. Consequently, it is 
not possible to determine (with any precision) the relative importance of various design options on the 
reliability and robustness of the aerospace vehicle. In addition, with no measure of reliability it is unlikely 
that the level of reliability and performance will be consistent throughout the vehicle. That situation can 
lead to excess weight with no corresponding improvement in overall reliability. Moreover, the factor of 
safety approach is logically inconsistent. It attempts to scale conditions using a mean or “worst-case” 
condition. In reality, a worst-case condition is rarely identifiable. 


Probability 

density 


Resistance 

^/strength) 



Failure 
(overlap region) 


Load or resistance 


Figure 4. Reliability-based design approach. 


In contrast to the traditional design procedure shown in figure 3, figure 4 illustrates how' uncertainties 
are handled in the reliability-based design approach. Here both the load and the strength are characterized 
by probability density functions. These distributions are due to uncertainties in the loads applied to the 
system (or subsystem) and to the strengths of different realizations of the system. The overlap region 
(where the load exceeds the strength) indicates the probability of failure. Note that for design of systems 
with small probabilities of failure, the tails of both the load and the strength distributions are the most 
relevant. Reliability-based design methods have been used within civil engineering (Sundararajan 1995) 
for decades 1 and in aircraft engine design (Cruse 2001 ) for about a decade. Applications to the structural 
design of airframes are only now- starting to emerge. Only academic studies of reliability-based design 
methods within the aerodynamics and controls disciplines are known to the authors. 


1 Civil engineering projects are generally designed using standard design codes. Many of these codes contain factors 
that can be adjusted based on the likelihood of occurrence of high loads or critical events, such as an earthquake of a 
specified magnitude. These factors provide the target reliability. The probabilistic aspects of these design codes may 
be hidden from the designer. 
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Newly emerging uncertainty-based design procedures will help to overcome the shortcomings of the 
traditional design procedures. In particular, measures of reliability and robustness will be available during 
the design process and for the final design. This information will allow the designer to produce a 
consistent level of reliability and performance throughout the vehicles — with no unnecessary over- 
designs in some areas. As a result, designers will be able to save weight while maintaining reliability. In 
addition, with an uncertainty-based design procedure it will be possible to determine the sensitivity of the 
reliability to design changes that can be linked to changes in cost. As a result, it will be possible to make 
trade-offs between reliability and cost. For the same cost, airframes can be made safer than with 
traditional design approaches — or, for the same reliability, the airframe can be made at a lower cost. 

This white paper consists of a survey of the state of the art in uncertainty-based design together with 
recommendations for a Base research activity in this area for the NASA Langley Research Center. In 
particular, section 2 provides a generic overview of current uncertainty-based design methods. Section 3 
focuses on uncertainty-based methods for aerospace vehicle design, describing the status of these methods 
and the barriers to their adoption. Section 4 proposes some specific research objectives with both short- 
term and long-term impact on industry and NASA design processes. Section 5 discusses the expected 
results from uncertainty-based design research. Our discussion is limited to the disciplines of 
aerodynamics, controls, structures, and systems analysis. Although other disciplines are excluded from 
the discussion, the reader will note that the authors have a diverse background, not just in terms of their 
disciplinary heritage but also in terms of their particular penchants towards experimentation, methods 
development, or applied computation. This diversity, combined with the relatively recent interest of most 
of the authors to this particular field of uncertainty-based design, has inevitably led to different emphases 
in the different disciplinary sections of this report and perhaps to occasional inconsistencies in 
terminology. The authors' views on this challenging subject are a work in progress. They have chosen to 
put their current thinking out for comment now rather than wait the several years necessary for a full 
meeting of their minds. 


2. Overview of Available Uncertainty-Based Design Methods 

The use of uncertainty-based design methods requires that the various uncertainties associated with the 
design problem be characterized and managed, and that the analysis and optimization methods 
incorporate this characterization of the uncertain quantities. In section 2.1 we focus on the 
characterization and management of uncertainties, and in section 2.2 we focus on the use of uncertainties 
in design. 


2.1. Characterizing and Managing Uncertainties 

Characterization and management of uncertainties is required at the individual discipline level as well 
as at the integrated, system level. These involve the computational and experimental uncertainties 
produced by the discipline analysis methods themselves, and the relationship of the uncertainties affecting 
the input with the uncertainties affecting the output of the methods. 

2.7.7. Computational Uncertainties 

In the context of computational modeling and simulation, two complementary categorizations of 
uncertainties are useful. One categorization distinguishes between parameter uncertainties and model 
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form uncertainties. 2 Parameter uncertainties , sometimes referred to as parametric uncertainties or 
parameter variability, are those uncertainties associated either with the input data (boundary conditions or 
initial conditions) to a computational process or with basic parameters that define a given computational 
process, such as the coefficients of phenomenological models. Mode I form uncertainties , sometimes 
referred to as structural uncertainties, nonparametric uncertainties, or unmodeled dynamics, are associated 
with model validity, i.e., whether the nominal mathematical model adequately captures the physics of the 
problem. 

Oberkampf et al. (1998) categorized uncertainties into three distinct classes. Variability refers to “the 
inherent variation associated with the physical system or the environment under consideration." 
Uncertainty is “a potential deficiency in any phase or activity of the modeling process that is due to lack 
of knowledge." Error is “a recognizable deficiency in any phase or activity of modeling and simulation 
that is not due to lack of knowledge;" an error may be either an acknowledged error or an 
unacknowledged error . The value of this categorization is that the approaches to characterizing and 
managing uncertainties are considerably different for the three classes. We are not going to emphasize 
this distinction in this white paper, and the interested reader should consult Oberkampf et al. (1998) for 
more information. In the material that follows, we shall typically use uncertainty in the more general 
sense. Whenever we mean the more specialized definitions of this paragraph, we shall use italics for the 
terms. 

In the context of a computer code used for computational modeling and simulation, the parameter 
uncertainties can be specified by means of interval bounds, membership functions, or probability density 
functions, as illustrated in figure 5. Interval bounds should be used when only the crudest information is 
known. They use only an upper and a lower bound for the parameter value; norm-bounded uncertainties 
are a special case. Probability density functions are the most detailed description. Membership functions, 
which are used in fuzzy logic approaches, provide an intermediate level of detail. The parameter 
uncertainty characterization can be based on expert opinion, experimental data, analytical estimates, or 
results from upstream computational processes. A more detailed discussion of the general mathematical 
framework for characterization of uncertainties from an engineering perspective can be found in 
Oberkampf, Helton, and Sentz (2001). 


Interval bound 


Uncertain variable 


Membership function 

Possibility 



Probability density function 

Probability 

density 



Figure 5. Uncertainty descriptions. 

Strategies for characterization of model form uncertainty are far less well developed than those for 


2 The terms model uncertainty and modeling uncertainty are occasionally used in the literature to mean the 
uncertainty in the model arising both from the model form and the model parameters. These terms should not be 
confused with the term model form uncertainty as we are using it here. 


7 



parameter uncertainties. Two general references of note are Beck (1987) and Draper (1995). A short 
summary of the Bayesian approach to model form uncertainty, together with some additional references, 
is given by Alvin et al. (2000). There do not appear to be any systematic applications of model form 
uncertainty characterization strategies to airframe design applications. 

Few systematic approaches are available for managing uncertainties in computational simulations. 
Adaptive mesh refinement is the most well developed, but most techniques are based on heuristic rather 
than rigorous approaches. Moreover, adaptive mesh refinement is limited to just one aspect of the 
uncertainty in the output of a code (discretization error) and it does not address the uncertainty in the 
model output due to uncertainty in the model input. 


2.1.2. Experimental Uncertainties 

The traditional approach to account for uncertainties in structural design is to introduce so-called 
design or safety factors on the loads and statistically based material properties. Thus, in a classical 
deterministic analysis, all the uncertainties are accounted for in a “lump-sum” fashion by multiplying the 
maximum expected applied stress by a single safety factor, e.g., 1.5. The specification of safety factors is 
generally based on empirical design guidelines established from years of structural testing and experience. 
Statistically based material properties are determined from a series of coupon tests. Design verification is 
achieved through testing by applying the worst-case loading condition to the structure and testing to 
failure. 

Uncertainties can be accounted for using probabilistic methods. Techniques for deriving probabilistic 
information and for estimating parameter values from observed data are found in the methods of 
statistical inference in which information obtained from sample data is used to make generalizations about 
populations from which the samples were obtained. Traditional methods of estimation include point and 
interval estimates. The common methods of point estimation are the method of maximum likelihood and 
the method of moments. Interval estimation includes determining the interval that contains the parameter 
value and a prescribed confidence level. However, when population parameters are estimated based on 
finite samples, errors of estimation occur. Explicit consideration of these errors is accounted for in the 
Bayesian approach to estimation. With this approach, subjective judgments based on intuition, 
experience, or indirect information are incorporated with observed data to obtain a balanced estimation. 
Validity of assumed uncertainty distributions is verified or disproved statistically by goodness-of-fit tests, 
such as chi-squares or Kolmogorov-Smimov methods. 

For situations in which sample data necessary to quantify parameter uncertainties is limited or 
nonexistent, fuzzy set (possibilistic) analysis can be used to account for uncertainties. Uncertainties are 
introduced by specifying membership functions. 

Systematic procedures for characterizing and managing' uncertainties in experimental activities 
include design of experiment methods (Taguchi 1987) and statistical process control techniques (Wheeler 
and Chambers 1992). The former focuses more on characterizing the uncertainties and the latter more on 
managing them. 

2.1.3. Uncertainty Analysis 

The objective of uncertainty analysis (or uncertainty propagation) is to characterize the uncertainties in 


’ Although in this context the word "control" is more conventional, we prefer the term "manage” to prevent 
confusion with the "controls" discipline. 
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the system output given some knowledge of the uncertainties associated with the system parameters 
together with one or more computational models and, ideally, some experimental data. This subsection 
will provide an overview various approaches to estimating the effects of parameter uncertainties; 
approaches to model form uncertainties are far less developed. Several methods are available for 
estimating the uncertainties in the performance predictions due to parameter uncertainties. They depend, 
of course, on whether the parameter uncertainties are specified in terms of probability density functions, 
membership functions, or interval bounds. 

2.1.3. 1. Probabilistic Analysis. The most well developed approaches to uncertainty analysis are based on 
parameter uncertainties specified in terms of probability density functions (PDFs). Uncertainty-based 
design methods using detailed PDFs are generally referred to as probabilistic methods. In this case, the 
uncertainty computing engine produces the nominal value of the performance functions as well as their 
PDFs. This process is typically completed by using some type of Monte Carlo or statistical sampling 
method. The concept is straightforward; the process model is invoked repeatedly for deterministic 
analyses performed for a set of input parameters generated according to their PDFs to produce a set of 
output samples. The statistical properties of the output performance functions are then deduced from the 
output samples. This approach is usually referred to as a simulation method or a sampling method. See 
Melchers (1999) for a general overview'. The simplest approach is the basic Monte Carlo method, in 
which the sampling points are drawn strictly according to the PDFs of the input parameters. Construction 
of accurate output PDFs can easily take thousands or even millions of simulations. Two of the more 
popular alternatives to the basic Monte Carlo method are Importance Sampling and Latin Hypercube 
Sampling (McKay, Beckman, and Conover 1979). The former enables accurate estimates of the tails of 
the PDFs; it is most useful in reliability-based design problems. The latter provides samples that ensure 
coverage of the full range of the input parameters with far fewer simulations than the basic Monte Carlo 
method. However, the tails of the output PDFs are generally quite inaccurate; it is most useful in robust 
design problems. Yet another refinement is Directional Sampling, which is useful for concentrating the 
sampling on a subset of the parameters of particular interest. At least a half-dozen general-purpose 
commercial software packages are available to serve as probabilistic uncertainty computing engines. 

A different approach to probabilistic uncertainty analysis is by the solution of stochastic differential 
equations. With this approach, the uncertainties may be associated with initial conditions, boundary 
conditions, transport properties, and source terms. Wiener (1938) developed a method of representing the 
associated randomness in the solution with an expansion in orthogonal polynomials in which the burden 
of representing the random component is carried by the polynomials; the coefficients of the expansion are 
smooth functions. Ghanem and Spanos (1991) and Ghanem (1999) refined Wiener's method and applied 
it to structural analysis problems. Xiu and Karniadakis (2001) have generalized the class of expansion 
functions and applied it to fluid mechanics and fluid-structures problems; they have shown that the set of 
Wiener-Askey polynomials contains an appropriate (and rapidly convergent) polynomial family for many 
of the PDFs of physical interest. This approach is referred to as Polynomial Chaos . One obtains the PDFs 
of every component of the solution at every space-time point. The cost is typically one to two orders of 
magnitude greater than that of a deterministic solution for the differential equation, yet is still orders of 
magnitude cheaper than required for an ensemble of solutions generated by a Monte Carlo method. 

2. 1.3.2. Fuzzy Logic. When the parameter uncertainties are characterized by membership functions, fuzzy 
logic is the basis for assessing the uncertainties in system output. Fuzzy logic allows one to create models 
based on inexact, incomplete, or unreliable knowledge or data, and, moreover, to infer approximate 
behavior of the system from such models. Fuzzy logic provides a computational engine to process these 
models (Harris, Moore, and Brown 1993). To date, fuzzy logic has found many applications in controls, 
manufacturing, pattern recognition, and even finance (Holmes and Ray 2001; Ham, Qu, and Johnson 
2000; Inoue and Nakaoka 1998; Isermann 1998; Dexter 1995; Hung 1995; Lim and Hiyama 1991; Lee 
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1990; Box and Draper 1987). Fuzzy logic is an extension of multivalued logic. Its predicates can be both 
crisp (not fuzzy), such as numbers, or true and false, and non-crisp (fuzzy), such as greater than, small, 
etc. Uncertainty characterizations using membership functions are sometimes referred to as possibilistic, 
which is often used to contrast this approach with the probabilistic approach. 

The central concept of fuzzy logic is the membership function, which represents the degree of 
membership of the fuzzy variable within the fuzzy set (Harris, Moore, and Brown 1993). The membership 
function may be thought of as a possibility function in contrast to the probability density function in 
probability theory. In fuzzy logic, input and output mappings are established through fuzzy algorithms 
based on a collection of rules, in the form of conditional statements. The composition of an input 
variable(s) with the fuzzy rules (relations) produces a fuzzy set for the output. To obtain a crisp output, 
the fuzzy set has to be de-fuzzified to a single value; different techniques can be used to accomplish this 
task. 

Fuzzy logic is appealing for its simplicity of application and implementation. It may be useful in 
applications wherein accurate physical models are not available, for preliminary design and analysis, or in 
cases wherein crude or fuzzy inferences and actions are acceptable. However, in uncertainty-based design 
and analysis, the use of fuzzy logic may be confined to the conceptual design and early preliminary 
design stages; the level of accuracy required in most aerospace applications in late preliminary and 
detailed design is far too stringent to allow for fuzzy logic. 

2. 1.3.3. Interval Analysis . Interval analysis was initially developed to account for errors in floating point 
arithmetic. The chief distinguishing element of its framework is that variables are represented by two 
scalars: a lower bound value and an upper bound value. These numbers reflect a measure of uncertainty in 
the knowledge of the actual value of the variable. Interval arithmetic involves the rules developed to 
perform mathematical operations with interval numbers. Interval analysis is ideally suited to deal with 
parametric uncertainties in systems. In its simplest form, a combinatorial analysis is used to construct 
interval distributions of the performance quantities. It has been applied in a variety of fields, including 
structural analysis and dynamic analysis, to accommodate uncertainties in parameters (Piazzi and Visioli 
2000; Jaulin and Walter 1997; Oppenheimer and Michel 1988; Ugur Koyluoglu and Elishakoff 1998; 

Qiu, Chen, and Elishakoff 1995; Rao and Berke 1997). However, these applications involve mainly 
simple problems of small order. Applications to medium-large size problems are lacking because by 
nature, interval arithmetic produces potentially conservative results, i.e., the results are not practical. The 
level of conservatism may be reduced if interval variables and arithmetic are used in closed-form 
solutions of the problem output, as opposed to throughout the numerical solution. Unfortunately, closed- 
form solutions are rarely available, except for the simplest of problems. Another minor problem with 
interval analysis is that it is computationally more expensive than traditional mathematics since it deals 
with intervals and not just scalars. However, with recent advancements in computing technology, this 
higher computational expense is not an issue anymore. In fact, interval arithmetic is accommodated in the 
latest SUN Microsystems Forte Fortran 95 Compiler. One other potential problem is that interval analysis 
may not be amenable to accommodating correlated uncertainty distributions. This issue may lead to 
additional conservatism in the predictions of bounds for the system output. 

2.1.4. Sensitivity Analysis 

Sensitivity analysis is related to, but different from, uncertainty analysis. The recent book edited by 
Saltelli, Chan, and Scott (2000) provides thorough coverage of this vast subject. The overview chapter by 
Campolongo et al. (2000) defines sensitivity analysis as “the study of how the variation in the output of a 
model (numerical or otherwise) can be apportioned, qualitatively or quantitatively, to different sources of 
variation, and of how the given model depends upon the information fed into it.” As used here, sensitivity 
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analysis ascertains which parameters have the greatest influence on the system output. It is a deterministic 
question. Its primary relation to uncertainty analysis is to enable the identification of which parameters 
really matter — conducting an analysis of the impact of the uncertainty of a parameter that has an 
insignificant effect upon the performance measures is futile. Given the high cost of most uncertainty 
analysis methods, it usually pays to first conduct a sensitivity analysis to weed out consideration of 
unimportant parameters. This process is often “step 0” in the construction of a response surface. 

Campolongo et al. (2000) list three general approaches to sensitivity analysis with numerical models. 
Factor screening determines which parameters (or groups of parameters) have the greatest impact on the 
model output variability. Various design of experiment techniques are typically employed. These consist 
of evaluating model output at the extreme values of the ranges of the parameters. Local sensitivity 
analysis utilizes first-order derivatives of model output quantities with respect to the parameters. Like 
factor screening, local sensitivity analysis is a deterministic calculation, but it is usually performed for a 
nominal set of parameter values. Some computational tools provide accurate and efficient sensitivity 
derivatives, but in most cases costly (and sometimes inaccurate) finite-difference approximations are used 
to compute the derivatives. Global sensitivity analysis typically uses statistical sampling methods, such as 
Latin Hypercube Sampling, to determine the total uncertainty in the model output and to apportion that 
uncertainty among the various parameters. 

If efficient sensitivity derivatives for the model are available, then local sensitivity analysis is by far 
the least costly in terms of computational time. At worst, its cost increases linearly with the number of 
parameters. However, it is limited to the vicinity of the nominal parameter values. The cost of factor 
screening methods that look at one factor at a time increases linearly with the number of parameter 
values, whereas methods that look at the full set of mutual interactions have an exponentially increasing 
cost. Statistical sampling approaches typically require thousands of evaluations of the computational 
model, although their cost is in principle independent of the number of parameters. This approach is 
prohibitively expensive for models with nontrivial computational times. The only feasible approach tor 
this important class of models is to generate an approximation to the model, and then to perform the 
statistical sampling on the approximation. Constructing such an approximation, however, is only feasible 
for a small number of parameters. 

2.7.5. Approximations 

Clearly, the methods described previously for uncertainty propagation and for sensitivity analysis are 
impractical when the computational time required to evaluate the process model is large. This problem is 
usually dealt with by using approximations. Several alternatives to sampling methods have been 
developed for estimating the PDF of the performance function. The Advanced Mean Value method (Wu 
and Wirsching 1989) exploits a first-order approximation to the performance function together with a Fast 
Probability Integration technique. The first-order approximation requires the computation of the gradients 
of the performance function with respect to all the random variables. The Fast Probability Integration then 
performs an integration over this linear approximation. Field, Paez, and Red-Horse (1999) have improved 
this method to yield more accurate moments of the PDF. Du and Chen (2000a) have developed a method 
based on Most Probable Point techniques for efficiently constructing the PDF. It, too, utilizes the first- 
order derivatives of the performance function. Both of these methods are usually implemented with finite- 
difference approximations to the gradients of the performance functions. Their cost can be dramatically 
lower if efficient methods for computing these derivatives are exploited. 

More generally, one can build an approximation, typically a response surface model, for the exact 
process model. The response surface is then used in lieu of the exact process model by the uncertainty 
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computing engine. Response surface approximations are constructed by the following three steps: 

• Experiment points are selected. 

• Response surface model order is selected. 

• Experiment points are fitted to the chosen model. 

The selected experiment points determine where in the design space the exact process model will be 
evaluated. Design of experiment techniques are typically employed; for an excellent treatment on the 
selection of the experiment points see Box and Draper (1987). When selecting the response surface model 
order, quite often either linear or quadratic surfaces are assumed. Once the model order has been 
determined, the response surface is fit using least squares. In some cases the model order is not selected 
beforehand; instead the response surface model is chosen such that the first two statistical moments of the 
response surface match that of the exact process model, resulting in first-, second-, or even higher-order 
surfaces. 


2.2. Analysis and Optimization Incorporating Uncertainties 

The previous subsection focused on methods for describing the uncertainties in the performance 
measures of a discipline or system. This description may be in the form of probability density functions, 
membership functions, or interval bounds. In this subsection, we discuss methods that use this 
information in design, e.g., by computing failure probabilities, by design of control systems, or by 
optimizing a system in the presence of uncertainties. 


2.2.1. Impact of Uncertainty on Performance Measures 

A type of performance measure that is pervasive in reliability-based design is a limit state function. 
Limit state functions are generally nonlinear relationships (constraints in the optimization context) used to 
define system failure conditions. Examples of the engineering quantities that may be embedded in limit 
state functions are stress, dynamic stability, temperature, fracture, or buckling. Conventionally, the failure 
domain is described by g(.r) < 0, where g is the limit state function. Figure 6 is an example of a limit state 
function for a problem with two design parameters. 



To date, limit state functions have been used exclusively with probabilistic uncertainty analysis. 
Mathematically, the failure probability P F is defined as 
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where f x (x) is the joint probability density function of X and the integration is carried out over the entire 
failure domain. For many engineering applications, solving equation (1 ) may entail a substantial 
computational effort. Some of the difficulties in evaluating the right-hand side of equation ( 1 ) include: 

• High dimensionality of the design space makes the integration very costly 

• Mathematical and computational complexity of the domain boundaries given by g(x) = 0 

• Lack of information regarding the joint probability density, f x (x) 

Because of these complexities, exact solutions of failure probabilities for general systems with 
arbitrary parameter uncertainty PDFs are not feasible; therefore, efficient numerical methods for 
computing approximate failure probabilities are required. In fact, the development of efficient 
approximate numerical techniques has been the subject of much research. 

The goal of First-Order Reliability Methods (FORM) and Second-Order Reliability Methods (SORM) 
is to compute failure probabilities efficiently by exploiting approximate forms of the limit state function. 
FORM and SORM replace the limit state function in equation (1) with first-order and second-order 
approximations, respectively. FORM and SORM reliability methods consist of four basic computational 
steps: 

1. Transform from physical space to standard normal space. 

2. Determine the most probable point (MPP). 

3. Approximate the limit state function at MPP. 

4. Compute the failure probability using the approximated limit state function from step 3. 

Step 1 is particularly important because of the properties of standard normal space. The most 
important property of standard normal space in regards to computing failure probabilities is that 
probability densities decay exponentially with the square of the distance from the origin. Therefore it one 
approximates the limit state function in the vicinity of the MPP (the closest point to the origin), the 
majority of the failure probability will be captured by an approximate limit state function because it is 
most accurate in the region that contributes the most to the integration. The specifics for computing 
failure probabilities for FORM and SORM differ slightly, but the primary difference between them is the 
order of the approximated hypersurface. In FORM, the limit state function is approximated by a tangent 
hyperplane at the MPP; in SORM, a quadratic hypersurface is used. This difference generally results in 
SORM producing more accurate estimates of failure probabilities, but at the cost of greater computer time 
due to the second-order gradient computations. 

Rackwitz (2000) has written a detailed review of FORM and SORM methods. Thacker et al. (2001) 
have provided a useful description of the weaknesses of current FORM and SORM algorithms. Thacker et 
al. (2001) note that Importance Sampling Methods are often used to increase the accuracy of the limit 
state function probabilities near the MPP. Choi and Youn (2001) have recently developed an alternative 
approach to reliability analysis that inverts the objective and constraint functions in the optimization used 
to identify the MPP. 

2.2.2. Bounded Uncertainty Design and Analysis 

The controls community has developed a special approach that they refer to as bounded uncertainty 
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design and analysis. In this approach, system uncertainties can be either parametric or nonparametric. 
Furthermore, the uncertainties can be real or complex time-independent functions, or linear/nonlinear 
time- varying functions (Balas and Packard 1996; Doyle, Wall, and Stein 1982). No matter what form of 
uncertainty one is dealing with, the uncertainty is assumed to be norm bounded, i.e., the maximum 
excursions of uncertainties are assumed to be known a priori. Figure 7 is an example of how the bounded 
uncertainty is added to a system. In this figure, P is the plant, d represents the exogenous inputs (typically 
unknown disturbances or reference signals), y p are the sensor measurements, A is the uncertainty block, 
and W and Z are used to define the interconnection between the plant and the uncertainty block. 
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Figure 7. Bounded uncertainty structure. 

The main objective in the Bounded Uncertainty Design and Analysis approach is to obtain bounds on 
the potential variations in system performance based on the bounds on the parametric and nonparametric 
uncertainties. The type of analysis required in this approach can vary significantly from one application to 
another. It can be simply a norm-based analysis in which the norm bounds at the uncertainty levels are 
propagated through to the system performance, e.g., bounds on stress levels due to bounded uncertainty in 
the material properties. Substantial work has been done for linear time-invariant dynamical systems in 
this area (Balas and Packard 1996; Doyle, Wall, and Stein 1982; Moser 1993; Balas and Doyle 1994; 
Gagnon, Pomerleau, and Desbiens 1999; Bendotti and Beck 1999; Nishimura and Kojima 1999; Pannu et 
al. 1996; Cheng and De Moor 1994; Balas et al. 1998). Various techniques are available to incorporate 
parametric and nonparametric uncertainties within the linear system. Both structured and unstructured 
uncertainty models can be considered in this framework (Balas and Packard 1996). A structured 
uncertainty model denotes a model in which uncertainty is directly associated with specific parameters in 
the system, while in the unstructured uncertainty model, uncertainties are general, and hence are not 
associated with any specific parameter in the model. Several methods are available for analysis of 
dynamic stability and performance of these systems. An example is the Linear Matrix Inequality 
framework, which uses convex programming (Scherer, Gahinet, and Chilali 1997; Apkarian and Adams 
1998). The treatment of linear time-invariant systems with uncertainty can be extended to linear time- 
varying systems as well as a certain class of nonlinear systems (e.g., linear parameter varying systems) 
with the aid of the Linear Matrix Inequality framework (Scherer, Gahinet, and Chilali 1997; Apkarian and 
Adams 1998). For general nonlinear systems, the treatment of bounded uncertainties is somewhat ad hoc , 
as there are no formal methods for dealing with such systems. However, in some cases, Lyapunov 
stability theory and norm-based propagation can be used to establish bounds on the performance of the 
nonlinear system. 
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2 . 2 . 3 . Optimization Under Uncertainty 

Several design optimization methods include uncertainty. For the purpose of this paper, the methods 
can be divided into three groups: sampling methods, robust optimization, and optimization for reliability. 
Sampling methods can be used to solve either robust optimization or reliability-based optimization 
problems, but we find it useful to discuss these methods separately. The sampling methods perform all 
experiments (whether mathematical simulations or physical tests) simultaneously and then optimize the 
design based on the results of those experiments. Robust optimization methods use the numerical 
optimization procedure to specify which simulations are needed and evaluate those simulations one at a 
time. The optimization for reliability methods also use numerical optimization procedures but with the 
goal of reliability rather than robustness. 

2.2.3.I. Sampling Methods . Sampling methods are based on the assumption that the designer will choose 
an optimum design by evaluating a performance or cost measure at a large number of points in design 
space. Ideally, these points define a smooth hypersurface with a well-defined feasible region and a single 
optimum point. If the cost measure (also called objective function) includes parametric uncertainty, then 
many evaluations are required at each point in design space to accurately characterize the hypersurface. 

Trouble arises when the value of the objective function is evaluated with a variety of different 
computer simulations and experimental data. Each source of measured and computed data includes errors 
(e.g., due to simplification in the mathematical models) and uncertainty (e.g., due to the difference 
between the ensemble averages and the individual test results). For a good overview of available sampling 
and response surface approximation techniques see Robinson (1998). For a discussion of the types of 
variabilities , uncertainties , and errors found in simulation codes and for uncertainty estimation methods 
see Alvin et al. (2000). 

Conceptually, optimization that includes uncertainty can be achieved by fitting all available data with 
some smooth surface and then using a mathematical optimization procedure to determine the best design. 
All sampling approaches considered here use the following three steps: (1) sample the design space, (2) 
approximate the objective function, and (3) estimate the optimum of the approximate surface. These steps 
can be repeated one or more times on smaller and smaller neighborhoods surrounding the current 
optimum point. 

Several popular techniques differ in steps (1) and (2). For example, the design space can be sampled 
using Taguchi arrays, Latin hypercubes, random points, or some subset of available data. Similarly, the 
approximate surface can be constructed using neural nets, polynomials, or splines under tension. The 
choice of method depends on knowledge of the design space (e.g., the objective function may vary 
linearly with respect to some design variables) and on the amount of available data. 

Response surface methods (RSM) and Taguchi parameter design methods are the most commonly 
used procedures. Venter and Haftka (1999) explain that RSM reduces the computational burden and 
simplifies the integration of optimization and analysis codes. Roy (1990) and Nair (1992) discuss the 
Taguchi method and some of its known strengths and drawbacks. Unal, Stanley, and Joyner (1993) 
provide a good tutorial on Taguchi methods written from an engineering perspective. The principal 
limitations of both RSM and Taguchi parameter design methods are: neither constraints nor multiple 
objective functions are accommodated; dimensionality restricts the application of these methods to 
problems with a small number of parameters and design variables; and they are far less applicable to 
uncertainty and error than to variability (as these terms are used by Oberkampf et al. 1998). Although 
Taguchi methods are effective for many problems, work still needs to be done to develop alternative 
approaches that overcome these limitations. 
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2.2. 3.2. Robust Optimization Methods. For our purposes, robust optimization methods seek to improve a 
design by making it insensitive to small changes in the design values, and they exploit sequential 
numerical optimization techniques. One method for achieving robustness is to minimize both the mean 
and variance. Thus, the optimization problem is formulated as a multiobjective problem: 

min w l c(t,0) + w 2 a 2 (t,6) ( 2 ) 

teQ’ 

where t are the design variables in the domain Q, 0are the uncertain parameters that are described by one 
or more PDFs, c is the mean of the objective function, cr is the variance of c due to the randomness of 0. 
and vv, are the user-defined weights. Depending on the choice of weights, this formulation will minimize 
the objective, minimize the variance in the objective, or discover some compromise between these two 
goals. 

Robust optimization methods are effective if c(t, 9) is a good simulation of the performance or cost 
goals and if the PDF of 6 is well characterized. For example, structural analysis codes can accurately 
predict the strains in metal truss structures given the loads. Moreover, the PDF of uncertain loading 
parameters such as wind speed or wave height have been collected. Therefore, robust optimization for 
sizing of civil engineering projects should result in safer and more cost-effective structures. 
Unfortunately, few simulation codes automatically predict the variance of the mean given a known PDF 
of the uncertain parameters. Tada, Matsumoto, and Yoshida (1988) provide a clear explanation of this 
method and its application to simple structural sizing problems. 

2.2. 3. 3. Optimization for Reliability. Optimization for reliability methods are based on the assumption 
that the design space is divided into two regions: success and failure. The goal of the optimization is to 
find the best design that is sufficiently far from the failure region so that the probability of failure is 
acceptably small. Thus, the optimization problem is formulated as a constrained optimization problem: 

min c(t,6) (3) 

t e Q 

subject to 

/ , [ l g(f,6>)<0]<r (4) 

where r is the reliability requirement, g(uO) = 0 defines the boundary (or limit surface) between success 
and failure, and P[g < 0] is the probability of failure for the current values of the design vector t. 

Optimization for reliability can be attempted using standard constrained nonlinear optimization 
procedures. The major difficulty is the computational expense of calculating the probability of failure. A 
secondary difficulty is that the constraint P F < r can be a highly nonlinear function of t even if g(t,6) is 
linear in t . 

The computational expense of optimization for reliability greatly hinders its acceptance. If an 
evaluation of the constraint g(t, 0) is computationally expensive or if the reliability requirement r is small, 
then Monte Carlo analysis can be prohibitively expensive since it requires thousands of evaluations of g at 
randomly sampled values of 6. An alternate approach is to (1) use the optimization code to find the most 
probable point of failure or MPP, (2) estimate the probability of failure using a linear approximation to 
the limit surface centered at the MPP, and (3) use the optimization code to minimize the objective for the 
required probability of failure. In this alternate approach, the optimization code is used twice, first to find 
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the values of 6 that define the MPP and then to find the values of t which reduce the objective without 
compromising safety. See Langley (2000) and Grooteman (1999) for a good overview of these methods. 


3. Current Status and Barriers 


3.1. Structural Analysis 

3. 1.1. Current Status 

Traditionally, the approach to designing aerospace structures with uncertainties is to use statistically 
based material properties (e.g., yield strength) and to introduce design factors. The statistically based 
material properties are characterized as A-basis and B-basis material property values (Anon. 1997). An A- 
basis material property is one in which 99 percent of the material property distribution is above the basis 
value with a 95 percent level of confidence. A B-basis material property is one in which 90 percent of the 
material property distribution is above the basis value with a 95 percent level of confidence. Design 
factors can be placed into two categories and include safety factors and design knockdown factors for 
stability critical structures. Safety factors account for uncertainties in a “lump-sum” fashion by 
multiplying the maximum expected applied stress by a single safety factor. The FAA air-worthiness 
certification requires a design safety factor equal to 1.5 for man-rated aircraft structures; however, safety 
factors as low as 1.02-1.03 have been used in the past for non-man-rated structures such as missiles. The 
safety factor is intended to account for uncertainties such as uncertainty in aerodynamic load definition 
and structural stress analysis, variations in material properties due to manufacturing defects and 
imperfections, and variations in fabrication and inspection standards. The safety factor is generally 
developed from empirically based design guidelines established from years of structural testing of 
aluminum structures. For a historical review of the evolution of the 1.5 factor of safety in the United 
States, see Muller and Schmid (1978). 

The traditional approach to designing thin-walled buckling-resistant structures is to predict the 
buckling load of the structure with a deterministic analysis and then to reduce the predicted load with an 
empirical design knockdown factor, which is intended to account for the difference between the predicted 
buckling load and the actual buckling load of the structure determined from tests. The differences 
between analysis and test results are mainly due to uncertainties in the structural geometry (e.g., 
imperfections), loading conditions, material properties, and boundary conditions. Design guidelines for 
stability critical isotropic structures can be found in several NASA documents including Weingarten, 
Seide, and Peterson (1968) for buckling of thin-walled circular cylinders, Weingarten and Seide (1968) 
for buckling of thin-walled truncated cones, and Weingarten and Seide (1969) for the buckling of thin- 
walled doubly curved shells. 

Many of the aforementioned design practices have been carried over to composite structures for lack 
of better design methods. In addition, these design methods can potentially result in overly conservative 
or unconservative designs of aerospace structures. Furthermore, these design guidelines do not include 
any data or information related to uncertainty sensitivity. 

3.1.1. L Probabilistic Analysis and Design Methods . A probabilistic design methodology reported in 
Anon. (1997) accounts for uncertainties in material properties; external or operational loads; 
manufacturing processes and their effects on material strength; environmental effects on strength such as 
moisture or radiation exposure; environmental history during operation; flaw and/or damage locations. 
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severity, and probability of occurrence and effects on strength; and predictive accuracy of structural 
models and analysis. The probabilistic approach uses the statistical characterization of parameter 
uncertainties and attempts to provide a desired reliability in the design. In the probabilistic approach, the 
uncertainties of the individual design parameters and loads are modeled by appropriate probability 
densities. These probability densities are combined into cumulative density functions by using 
transformation equations. In this case, the design parameters have an uncertainty that is quantified in 
terms of risk. The credibility of this approach depends on two factors: the accuracy of the analytical 
model used to predict the structural response, and the accuracy of the probabilistic techniques employed. 
This probabilistic methodology has shown some success in the design of composite structures where the 
parameter uncertainties are well-known. For example, the IPACS (Integrated Probabilistic Assessment of 
Composite Structures) computer code was developed at NASA Glenn Research Center (Chamis and 
Murthy 1991 ), and a probabilistic stability analysis for predicting the buckling loads of compression- 
loaded composite cylinders was developed at Delft University of Technology (Arbocz, Starnes, and 
Nemeth 2000). 

3.1. 1.2. Fuzzy Set or Possibilistic Analysis Methods. For situations in which sample data necessary to 
quantify parameter uncertainties are limited or nonexistent, fuzzy set analysis can be used to account for 
uncertainties. In these methods, uncertainties in input parameters (e.g., dimensions. Young’s modulus) are 
defined by membership functions. An example of a membership function is shown in figure 8. 

Possibility 



Figure 8. Example of membership function. 

The vertical scale is the possibility that an uncertain quantity takes on a given value, and the horizontal 
scale shows values of the uncertain quantity. The possibility varies from zero (no possibility) to one 
(maximum possibility). In this example, the most likely value of the uncertain quantity is 0.25. The 
uncertain quantity is bounded by 0.20 and 0.30 at Possibility = 0.0. The objective is to use the 
membership functions of the input parameters to determine the corresponding membership functions for 
the response quantities (e.g., stress, buckling load). The membership functions for the response quantities 
are then compared with the membership functions of the allowable responses to determine the possibility 
of failure. 

A subset of fuzzy set or possibilistic analysis is sometimes called interval analysis. In this approach, 
upper and lower bounds are placed on the uncertain input parameters, and the resulting upper and lower 
bounds are calculated for the response quantities. The objective is merely to bound the response rather 
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than to indicate the likelihood of the response taking on a given value. This approach is equivalent to 
working with membership functions at a possibility of zero. 

Fuzzy set or possibilistic analysis methods have been proposed by Noor, Starnes, and Peters (2000) 
and have been applied to structural problems such as the probabilistic strength predictions of bonded 
joints by Stroud, Krishnamurthy, and Smith (2001). In addition, interval analysis has been used to predict 
response bounds for compression-loaded composite shells with random imperfections, e.g., Hilburger and 
Starnes (2000). 

3. 1.1.3. Example. The example illustrated in figure 9 (from Fadale and Sues 1999) illustrates how 
quantitative design for reliability enables effective trade-offs between weight and reliability. An existing 
lap joint connecting two integral airframe panels was redesigned for minimum weight and to meet 
reliability requirements. Failure analyses (the limit state functions) involved fatigue and discrete source 
damage. Reliability-based design methods yielded an improved design that saved 19 percent in weight 
while providing the same reliability. In addition, this method provided sensitivity information that showed 
how weight varies with reliability. In this case, small increases in weight produced large increases in 
reliability. Weights are shown normalized w ith respect to the weight of the original joint. 
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Figure 9. Reliability versus weight trade-off (from Fadale and Sues 1999). 


3.1.2. Barriers 

Most of the barriers to the use of uncertainty-based design methods in the structures discipline are 
shared with the other design disciplines. 

Bl. Industry feels comfortable with traditional design methods. 

There is always inertia associated with existing processes. Certainly, traditional design methods have 
been very successful in the aerospace industry. Nevertheless, the traditional methods and processes are 
characterized by costs, schedules, and performance limitations that are significant problems in the modern 
business climate. A necessary condition for overcoming this barrier is ensuring that the transition by 
industry to the new uncertainty-based methods and processes is relatively painless. 


B2. Few demonstrations of the benefits of uncertainty-based design methods are available. 
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The technology advocates of uncertainty-based design need to provide compelling demonstrations of 
the benefits of uncertainty-based design methods and a compelling business case for changing to the new 
processes. This case must demonstrate measurable benefits and must be couched in terms readily 
understandable by both technology managers and practitioners. Given the head start of the structures 
community on uncertainty-based design methods, this barrier is even greater for the other design 
disciplines. 


B3. Uncertainty-based design methods are more complex and considerably more expensive than 
deterministic design methods. 

Advances in computer hardware and software, especially in automation, are bound to help. However, 
orders of magnitude improvements are needed to overcome this barrier. Fundamental breakthroughs are 
needed in this area. 


B4. Characterization of structural imperfections and uncertainties necessary to facilitate accurate analysis 
and design of the structure is time-consuming and is highly dependent on structural configuration, 
material system, and manufacturing processes. 


To some extent this structures-specific barrier will naturally lessen with time as more and more such 
characterizations are done. However, methods must be established for generalizing the specific results 
that come out ot tests of particular configurations, materials, and manufacturing processes. 


3.2. Aerodynamic Testing and CFD 

3.2. 1. Current Status 

For the purposes of discussing the risk associated with using the results obtained from experimental 
and/or computational aerodynamic simulations, it is convenient to think of a simulation as a single pass 
through a manufacturing process (Eisenhart 1969). The outputs of the “manufacturing” process are the 
numbers generated either by the instruments in an experiment or by the computer codes in a computation. 
It is the uncertainty (fuzziness) associated with those numbers that a simulation contributes to the overall 
risk in the design of an aerospace vehicle. Hence, the objective of any aerodynamic simulation effort must 
be to quantify the subprocess and overall process variation 4 and contain it to acceptable levels in the 
simulation results and thence into the final design. 

Wheeler (1990) discusses the three stages involved in quantifying and containing variation in the 
making of an industrial product. These stages are useful for discussing the risk associated with both types 
of aerodynamic simulation: testing and CFD. The manufacturing stages, as presented by Wheeler, 
together with the analogous aerodynamic simulation stages are given in table 1. Usually, the three 
aerodynamic simulation stages are carried out simultaneously, at least in part (Rubbert 1998), but the 
strategies (objectives) of each for managing and minimizing risk are different. 


4 We will use the term "variation” to designate the scatter in replicated results. If the scatter is not know n and controlled, it is not 
possible to consistently and credibly assess and correct the offset of the mean of the results from the true value. 
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Table 1. Stages for Minimizing Risk Associated With Manufacturing and With Aerodynamic Simulation 


Manufacturing 

Aerodynamic simulation 

Stage 

Objective 

Stage 

Objective 

Product design 

Choose design parameters so that 
the product will be insensitive to 
variation in raw materials and 
manufacturing process conditions 

Airframe 

design 

Choose design parameters so that 
the simulation results will be 
insensitive to the simulation 
process 

Manufacturing 

process design 

Choose the manufacturing 
process conditions so that the 
product will be insensitive to 
variation in those conditions 

Simulation 
process design 

Design the simulation process to 
be insensitive to variation in the 
process parameters 

Manufacturing 

Control the manufacturing 
process to obtain desired product 
consistency 

Simulating 

Manage the level of variation so 
that the process variation has 
known and traceable bounds 


This subsection addresses previous work and the current status of work in the three stages of 
simulation as described in table 1. The various efforts can be divided into three types: general strategies, 
cul hoc “example” evaluations, and systematic evaluations. It is beyond the scope of this section to report 
and evaluate all of the work in these areas. However, we will list certain publications that illustrate what 
we believe to be the state of the art. 

Airframe Design, A usual design strategy is to choose airframe designs for which one has sufficient 
experience to be able to assess risk. In this capacity, CFD has made some inroads to the design process 
for over a decade (cf. Rubbert and Goldhammer 1989). Many issues still abound regarding the role of 
CFD in design (Rubbert 1990, 1994, 1998; Raj and Singer 1991, Raj 1998; Cosner 1994, 1995), not the 
least of which is the rigorous establishment of confidence in computational results. 

Beyond the usual design strategy of choosing airframe designs for which one has sufficient experience 
to be able to assess risk, the pioneering work in this area seems to be optimization based on statistical 
modeling of the design criteria (Huyse 2001). Such modeling produces “softer” but more robust optima. 
There does not appear to be any work in CFD that attempts to produce aerodynamic designs that are 
inherently robust in the face of input uncertainties and discretization and modeling errors. Also, neither 
general strategies nor systematic evaluations for airframe design have been developed. 

Simulation Process Design. General strategies have been developed for assessing and managing risk 
in both computational and experimental simulations. Examples of general strategies for computations are 
Cosner (2000), Mehta (1998), Oberkampf et al. (1998), Rizzi and Vos (1998), and Roache (1998). These 
strategies typically require (1) assessment and management of numerical error due to discretization of the 
governing differential equations and (2) a hierarchy of validation phases, primarily statistical comparisons 
with experimental results. These requirements are not unlike the conceptual distinction between CFD 
code “calibration” and “validation” from Waggoner et al. (1994) as part of an international effort to 
delineate a proper process for code validation and to provide a comprehensive series of test cases 
(AGARD 1994): 

CFD Code Calibration : The comparison of CFD code results with experimental data for realistic 
geometries that are similar to the ones of design interest, made in order to provide a measure of 
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the code’s ability to predict specific parameters that are of importance to the design objectives 
without necessarily verifying that all the features of the flow are correctly modeled. 

CFD Code Validation : Detailed surface- and flow-field comparisons with experimental data to 
verify the code’s ability to model accurately the critical physics of the flow. Validation can occur 
only when the accuracy and limitations of the experimental data are known and thoroughly 
understood and when the accuracy and limitations of the code’s numerical algorithms, grid 
density effects, and physical basis are equally known and understood over a range of specified 
parameters. 

With this distinction, many prior cases of purported code validation should be considered a collection 
of specific code calibrations. Furthermore, this distinction makes clear that computational validation can 
only occur in conjunction with rigorous experimental uncertainty management. Results from this 
distinction have been published by Bussoletti (1994). 

Promising efforts are underway by several research groups to reduce, manage, and assess the 
numerical error using adjoint equations (Giles and Pierce 1999; Habashi et al. 1998; Patera and Ronquist 
2001; Roberts, Sidilkover, and Thomas 2000; Venditti and Darmofal 2000). Such ideas seem to have 
come of age and the use of such methods appears to be spreading. However, systematic evaluations of the 
methods, especially regarding the scatter due to variation in code and observer, appear to be lacking. 
Validation efforts at this stage appear to consist mostly of example problems. The most complete effort of 
this type may be the work of Aeschliman and Oberkampf (1998). Others have attempted to assess 
variation across several codes using several experimental data sets (Barber et al. 1998; Elsholz 1997; 
Georgiadis, Yoder, and DeBonis 1999). 

Examples of general strategies for designing experimental simulations for managed risk are 
Aeschliman and Oberkampf (1998), Anon. (1995), and Mayo (1996). Overall, such strategies consist of 
one or more of the following three elements: (I) creation of a hierarchy of validation experiments, (2) 
evaluation of the effects of instrument errors using error propagation analysis, and (3) replications during 
the experiment to allow for statistical characterization of the data scatter for that test. Although it is clear 
that any competent research facility would follow these strategies, it is not at all clear how well the 
strategies can be counted on to fully characterize the “fuzziness” of the experimental results. The reason 
for this seems to be a general lack of systematic evaluation of the strategies themselves. 

Simulating. Approaches are in place for both experimental and computational aerodynamic 
simulation for estimating a portion of the uncertainty in the results of any given simulation effort. The 
methods described in the previous paragraph for design of an experimental simulation process are used 
widely in the industry for estimating the uncertainty of experimental results (Belter 1996; Cahill 1996; 
Meyn 1998; Kammeyer and Rueger 2000), although their general effectiveness is not well understood, 
primarily because systematic evaluations across facilities are essentially unavailable. The need for 
managing and evaluating numerical error in individual computational simulations is well-known and is 
required by archival publications. As described previously, methods have been developed for estimating 
such errors. However, systematic evaluation of such schemes is lacking. 

The only effort known to us that attempts to evaluate experimental simulation process as a process is 
that of Hemsch et al. (2000). The effort uses the methods of statistical quality control, check standard 
(surrogate) testing, and replicates during customer testing to evaluate and control the process for all tests 
conducted in a facility. The method is being used at Langley Research Center in eight wind tunnels and 
one nozzle test facility. We are not aware of any similar efforts associated with computational results. 
Moreover, the strategies have not yet been developed for evaluating and controlling processes from 
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multiple facilities to produce consistent data. 

Probably the leading work for addressing computational uncertainty as a process is being conducted 
by Oberkampf and his associates (Oberkampf and Blottner 1998; Oberkampf 1998; Oberkampf et al. 
1998; Oberkampf and Trucano 2000). Significant contributions can also be found in the work of Roache 
(1990, 1998). These works could be heavily leveraged toward the current interest in uncertainty-based 
design. 

3 . 2 . 2 . Barriers 

The main theme that seems to arise out of the previous discussion in this section is a lack of 
systematic, quantitative, and credible (traceable) evaluations of the available methods for minimizing, 
managing, and estimating the errors associated with the results of both experimental and computational 
simulation. Aerodynamics shares with structures the barriers of industry comfort with traditional design 
methods (Bl), few demonstrations of benefits of uncertainty-based design (B2), and increased complexity 
and greatly increased cost for uncertainty-based design methods compared with deterministic design 
methods (B3). Indeed, the cost increase is even more of a barrier in aerodynamics because the nonlinear 
aspects of computational aerodynamics make the computation of gradients much more of a challenge than 
for computational structures. 

3 . 2 . 2 . /. Statistical Process Control . Additional barriers include the dearth of statistical process control 
activity, the lack of any current approach for characterizing the model form uncertainties associated with 
transition and turbulence, and the lack of a viable approach for characterizing uncertainties in sensitive 
nonlinear problems. 

B5. There is a dearth of statistical process control activity in aerodynamics. 

This barrier afflicts most disciplines. In the case of aerodynamics, two principal causes, or “sub- 
barriers,” are involved: 

a. Present and projected CFD code calibration, verification, and validation activities are expensive and 
insufficiently general to be applicable for predictive accuracy assessment. 

b. Calibration and validation exercises need to include code developers (or analysts) and experimentalists, 
but these groups are not motivated to work together. 

High-fidelity CFD computations (Reynolds-averaged Navier-Stokes) are notoriously expensive — 
runtime can last upwards of a day, even on a 32-processor cluster, for a full aircraft computation. This 
expense precludes, for example, the practical use of even the most sophisticated sampling methods for 
assessing the effects of variability. Part of B5(a) is a reinforcement of B4. The additional aspect is that the 
calibration, verification, and validation approaches currently in use by the CFD community need to be 
reconciled with the general verification and validation framework that has emerged in the past few' years. 

The second aspect of this barrier, B5(b), involves cultural and financial forces acting in concert, and 
certainly results in a dearth of statistical process control activity. Consider the following institutional 
groups: 

Airframe Designers. These workers typically have little or no time to do any systematic work other 
than design. Their efforts are aimed at optimizing a design as fast as possible using both experimental and 
computational simulation. Unfortunately, their support groups involved in research, development, and 
testing have little funding with which to carry out evaluations that cross their institutional boundaries. 
Furthermore, any such systematic evaluations carried out are likely to be proprietary. 



Simulation Process Designers. These workers typically have little funding to do any systematic 
work. Their day-to-day pressures are likely to be problem solving and extension of simulation capabilities 
rather than systematic evaluation. Notable exceptions are the adjoint equation work for estimating and 
managing numerical errors and the statistical quality control work for estimating and controlling data 
scatter in wind tunnels. Both types of simulations suffer from a lack of definitive systematic hierarchical 
experiments with which to evaluate modeling errors. 

Simulation Users. Users of both types of simulations are under severe pressure to produce “good 
quality” results as quickly as possible for as little money as possible. Users very seldom have the time or 
funding to carry out systematic evaluations. They are typically quite willing, however, to carry out small 
efforts to help with such evaluations if they are organized and funded by others. A common language will 
need to be agreed to among the practitioners so that calibration, validation, verification, certification, 
uncertainty, and so forth have consistently defined meanings and contexts. 

Systems Approach. A systems approach could be a useful context to develop computational and 
experimental aerodynamic methodologies for risk based design. Here the focus is drawn from top-level 
system requirements, such as the successful insertion of the space shuttle into orbit (fig. 10). One could 
just as well focus on other missions. Examples would include (but not be limited to) planetary entry, such 
as the Mars Lander, or aircraft performance, such as commercial transport cruise and high lift 
aerodynamics. 



Figure 10. Systems approach to computational aerodynamics for uncertainty-based design. 

All of these examples share a number of features from a design process perspective. Perhaps foremost 
is the need for simulating or modeling flow environments that cannot be fully reproduced by conventional 
experimentation. This approach leads to the use of computational tools of varying fidelity, experimental 
processes of varying approximation, and so forth. A necessary consequence is the amalgamation of 
differing fidelity technologies for the extrapolation to system-level conditions for performance 
predictions. 

Easterling (2001) has proposed a process to quantify the uncertainty of computational predictions, and 
a summary figure from his report is presented in figure 1 1. The details of this work are quite extensive 
and beyond the scope of this white paper, but his approach could be influential in uncertainty-based 
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design activities. One key result here is the Quantified Prediction Uncertainty — a product of computation, 
experiment, and analysis — that is used to assess system performance requirements. 

Uncertainty quantification activities exist both among and within the elements of Easterling’s model. 
Note that he distinguishes the system environment from the testable environment , explicitly recognizing 
the extrapolative nature of many computational physics or ground-based testing environments (think of 
Reynolds Number vis-a-vis aircraft performance). 



Figure l 1 . Quantifying the uncertainty of computational predictions (from Easterling 2001 ). 

To view computation in the context of an overall design process, drawing on the work of Rubbert 
(1994) is useful. Figure 12 is taken from Rubbert as a characterization of the general design activity: rapid 
progress early in the design phase, asymptotic advancement late in the design activity, and an ever- 
reducing variation throughout the process. The range of usefulness for a particular CFD code is then 
shown in the context of this design process (fig. 13). Because of the inherent upstream limitations, such as 
setup time, and downstream limitations, such as accuracy, a hierarchy of methods is used through the 
design process: lower order faster methods early in the process and higher order slower methods late in 
the design process. This general reasoning would apply to other simulation technologies as well. 

Several things can be done to more rigorously manage this risk, and some examples are shown in 
figure 14 and figure 15. Activities ranging from adaptive gridding to implementing expert systems could 
significantly improve the risk management for the utilization of any particular code. A rigorous means to 
link codes of varying fidelity could also significantly impact the risk associated with a design process. 
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Figure 12. Variation associated with design process (from Rubber* 1994). 



Figure 1 3. Range of usefulness of code within design process (from Rubber! 1 994). 
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Figure 14. Within-eode enhancements for design process. 



Figure 15. Cross-code interaction to enhance the design process. 

J.2.2.2. Aerodynamics Model Form Uncertainty 

B6. Effective approaches for characterizing model form error are lacking. 

Undoubtedly the greatest challenge in characterizing the uncertainty associated with CFD 
computations is the characterization of the model form uncertainty. The aerodynamics models used in 
design include simple algebraic equations, linear aerodynamics (vortex-lattice and panel) methods, Euler 
methods, laminar Navier-Stokes methods, and Reynolds-averaged Navier-Stokes (RANS) methods. Most 
aerodynamics effects of design interest involve turbulent flow. Turbulence modeling has challenged 
ae rod^namicists for decades. CFD calculations that resolve all spatial and temporal scales require 
0(Re /4 ) storage and 0(Re U/4 ) operations (see, for example, Rogallo and Moin 1984), where Re is the 
Reynolds number. Since such computations for flows of aerodynamic design interest are well beyond 
foreseeable computer resources, phenomenological turbulence models are a necessity. For preliminary 
and detailed design, aerodynamic codes utilize various 0-equation, I -equation, 2-equation, algebraic 
Reynolds stress (second-order) closure, and full second-order Reynolds stress models. 
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The parametric uncertainties associated with the coefficients of a given turbulence or transition model 
can be (but almost never are) characterized with standard (but expensive) sensitivity analysis methods. 
The dearth of plausible strategies for characterizing the model form uncertainty associated with the 
structure of the turbulence model employed in RANS computations, as well as the simplifications 
associated with using laminar, Euler, linear, and algebraic aerodynamics models, inhibits many in the 
CFD community to tackle the more manageable aspects of computational uncertainty in CFD. The only 
work we have been able to find on this subject is by Coleman and Stem (1997). 

For some aerodynamics design applications, the challenge of predicting the onset of laminar-turbulent 
transition and the associated modeling of the transitional region is also considerable. Here, too, 
characterization of model form uncertainty is a significant barrier. The same can be said of the models 
used in Large-Eddy Simulation and Detached-Eddy Simulation. 

3.2.2. 3. Sensitive Nonlinearities 

B7. There are no dependable approaches to uncertainty quantification for nonlinear problems. 

Aerodynamics is replete with situations in which small changes in parameters lead to drastic changes 
in the flow' and therefore to drastic changes in the performance measures. Phenomena such as the 
transonic drag rise, flow separation and reattachment, shock-boundary-layer interactions, vortex bursting, 
limit-cycle oscillation, and boundary-layer transition suddenly change the whole character of the flow. 
Uncertainty characterizations performed for one flow' regime do not extrapolate well to another regime. 
Presently, no strategy can be used with confidence for characterizing uncertainties for sensitive nonlinear 
problems. This lack of confidence on the uncertainty bounds often leads to restricting the design space to 
linear regimes for which designers have sufficient experience to trust their judgment on the uncertainties. 
This restriction can lead to lost opportunities for higher performing vehicles. Although it is encouraging 
that work on uncertainty quantification for strongly nonlinear problems is expanding, much remains to be 
done. 

3.3, Control Systems 

3.3.1. Current Status 

Stability and performance robustness have long been considered significant issues in control system 
design and analysis. Researchers like Bode (1945), Nyquist (1932), and Evans (1948) were instrumental 
in developing some of the fundamental concepts for the analysis and design of feedback control systems. 
For the most part, their work focused on frequency domain graphical methods for single-input/single- 
output (SISO) systems. As a result of their efforts, quantifiable metrics for characterizing stability 
robustness were developed. Even today, stability robustness for SISO systems is typically characterized in 
terms of gain and phase margins. With the help of computers and powerful numerical software, the SISO 
concepts for analyzing robustness have been generalized to include multi-input/multi-output (MIMO) 
systems using loop-gain singular value analysis (Safonov 1977). Unfortunately, singular value analysis in 
its general form can, in many cases, result in very conservative designs that may not necessarily reflect 
the true nature of the parameter/model uncertainties. Although there has been much development since 
the introduction of singular values, e.g., structured singular-value (p) (Doyle 1982, 1983) and p -analysis 
(Weingarten and Seide 1969), the developments rely on frequency domain bounded uncertainty 
representations that may not adequately model the true nature of the uncertainty. In an attempt to 
overcome this limitation, researchers are now investigating methods for analyzing the stability of 
feedback systems with parameter uncertainties defined in terms of probability density functions. This 
approach is generally referred to as “probability of stability" (Stengel 1980; Lim and Junkins 1987; 
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Stengel 1991). 


One of the first papers to employ probabilistic descriptions of parametric uncertainties for the analysis 
of stability robustness was Stengel (1980) with a follow-up by Stengel (1991). In these papers, the author 
employs the Monte Carlo simulation method to assess the effect of probabilistic parametric uncertainties 
on system stability. For linear time-invariant systems, stability may be evaluated by computing the 
eigenvalues of the system. Eigenvalues with negative real parts are stable: those with positive real parts 
are unstable. Since the value of the real part of the system eigenvalues has only two possible outcomes, 
i.e., positive (unstable) or negative (stable), the probability of instability, ^[instability], may be defined 
mathematically as 


^[instability] = 1 - J f z {a)da (5) 

where a is vector of the real parts of the system eigenvalues and f z is their joint probability density 
function. Note that/j is almost never known analytically, but may be evaluated implicitly using Monte 
Carlo techniques. As an example, consider the characteristic equation for a typical second-order system 
under feedback control: 


.v 2 + 2£co + to 2 (6) 

Assume that the closed-loop system damping <; and natural frequency 0) are uncertain parameters with 
normal distributions. Specifically, let q have a mean of 0.707 with a standard deviation of 0.5 and co have 
a mean of 1.0 with a standard deviation of 0.2. A Monte Carlo simulation has been performed to assess 
the probability of instability. The results are presented in figure 16. In this figure, regions of highest 
eigenvalue density are shown in red and yellow and regions of lowest density in dark blue. Regions of 
zero eigenvalue density are shown in white. 
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Figure 16. Probability of instability. 

Using the data presented previously one can easily compute the probability of instability. In this 
example /^instability] = 7.82 percent. Lim and Junkins (1987) introduce a technique for approximating 


Eigenvalue distribution 
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the probability of stability using linear perturbations of system eigenvalues and techniques from 
probability theory. This method belongs to the class of response surface methods because of the 
assumptions used for the system eigenvalue models. Specifically, the assumption is that the eigenvalues 
inside the failure domain may be represented by a first-order Taylor series expansion about some nominal 
point. This method may prove extremely valuable for a system in which parameter uncertainties have 
small variances and eigenvalues with near-linear behavior. For systems that do not satisfy these 
requirements, more research is needed to quantify the impact of the underlying assumptions. 

Spencer (1994) and Spencer et al. (1992) explore the use of first-order and second-order reliability 
methods (FORM/SORM) for computing probabilistic stability/performance measures for structural 
control systems. This body of work addresses several interesting aspects of probability for controlled 
systems. Similar to the work of Stengel (1980) it addresses probability of stability using eigenvalue 
considerations, but extends the scope to include system RMS output response values and control system 
RMS input values, which are particularly useful metrics used in many control applications. In terms of its 
approach to the pure stability problem, i.e., eigenvalue considerations, it casts the problem as a classical 
reliability problem for a series of components with limit state functions made up of the individual 
eigenvalues. In series systems, if one of the components fails, the entire system fails. The analogy to 
stability is that if any one of the system eigenvalues has a positive real part then the entire system is 
unstable. An example of a family of three eigenvalue limit state functions with two uncertain parameters 
is given in figure 17. 



Figure 17. Eigenvalue limit state function. 

The examples presented by Spencer (1994) and Spencer et al. ( 1992) provide an informative view of 
reliability methods applied to the area of active structural control. The types of uncertainties addressed in 
these examples included stiffness, damping, inertia, actuator effectiveness, and time delay. Although 
these examples are limited to low-order structural systems with corresponding low-order compensators, 
the basic theoretical structure of FORM/SORM should be capable of handling higher order controlled 
systems. 

3.3. LI. Bounded Uncertainty Design and Analysis. A typical control design with bounded uncertainty 
structure is presented by Balas et al. (1998). In this application, linear control designs for the F-14 aircraft 
lateral-directional axis during powered approach to a carrier landing are presented. The controllers are 
designed using the structured singular value (p) framework (Balas and Packard 1996; Moser 1993; Balas 
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and Doyle 1994; Gagnon, Pomerleau, and Desbiens 1999). The performance objective is to design robust 
controllers such that the true airplane (nominal model plus uncertainty) responds effectively to pilot 
command in the form of lateral stick and rudder pedal inputs. A block diagram, representing the various 
interconnections between the nominal model, the controller K, and the uncertainty A G is presented in 
figure 18. The nominal model, designated as FI4 nom , has four states, lateral velocity v, yaw rate r, roll rate 
p, and bank angle (j). This model is assumed to be linear and time-invariant for a trim condition of 10.5 
degrees angle of attack and an air speed of 137 kn. For the purpose of control design, the aircraft is 
modeled as having three control inputs, differential stabilizer deflection 5 d(ab , rudder deflection 8 rud , and 
differential spoiler deflection S dsp . Three measured outputs are used in the feedback loop: roll rate, yaw 
rate, and lateral acceleration y ac . The dashed box in the figure represents the true airplane model, which 
includes the nominal model, actuator models, and W in and A G , which parameterize the uncertainty in the 
model. The model uncertainty A Cl assumed here is called input multiplicative plant uncertainty. The 
uncertainty is modeled as three individual, complex, full-block multiplicative uncertainties at the input of 
the aircraft model. These uncertainties accommodate errors in differential stabilizer, differential spoiler, 
and rudder moment coefficients. The reasoning behind using three individual blocks to represent the 
uncertainty is based on an assumption that isolated errors in differential stabilizer, differential spoiler, and 
rudder moment do not couple into each other. Two linear |i controllers were successfully designed, 
implemented, and tested in pilot-in-the-loop simulations at the crewed flight simulator at the U.S. Naval 
Air Warfare Center in Patuxent River, Maryland. The test pilots both concluded that the p controllers 
achieved all performance objectives, with a Cooper-Harper rating of 2 and 3, respectively. 


Mechanical connections 



Figure 1 8. Block diagram of FI 4 robust controller (from Balas 1998). 
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3.3. 1. 2. Fuzzy Control. Ham, Qu, and Johnson (2000) describe a robust fuzzy control development for 
robot manipulators that guarantees both global stability and performance. In the approach taken here, a 
robust suboptimal control is designed first and fuzzified for each rule to guarantee stability in each fuzzy 
set. Then, individual fuzzy controls are blended to form the overall fuzzy controller. The procedure for 
designing a robust fuzzy controller is given as follows: 

• Subsets F, are chosen according to their proximity to hyper-balls of the auxiliary state (c = z(x)), 
reflecting the designer’s choice for stability regions and performance. Membership functions 
M,(x) are chosen to make the sets F t fuzzy. These functions are required to have their degree of 
membership between zero and one. 

• Individual fuzzy controls are selected according to the fuzzy rule. 

o If x e F n then control is given by u =w,(x), where u,(x) is a fuzzy control which depends 
on the bounds on the nonlinear part of the state matrix and the auxiliary state vector. 

• The overall fuzzy control u t is obtained by blending the individual control laws according to the 
standard fuzzifying rule. 

This design procedure provides robustness, global asymptotic stability, and performance. It should be 
noted however that the robustness characteristic of this control approach is obtained from the Lyapunov 
stability theory, together with a norm-bounded approach to handle nonlinearities and uncertainties, and 
not as a specific implementation of the fuzzy set theory. The fuzzy control scheme was applied to control 
of a two degree-of-freedom robot manipulator. The bounds in the eigenvalues of the inertia matrix were 
chosen as 0.5 and 9, respectively. The nonlinearities in the state matrix are bounded from above by a 
quadratic function of the state vector norm. The desired trajectories for both joint angles are given by 
1 - cos(f). Three subsets of the state space F h i = 1 , 2, 3 are defined: the states at or close to the origin; 
states either on, inside, or close to the hyper-ball defined by ld = 0.005; and the states on the outside and 
not close to the hyper-ball. Figure 19 illustrates the triangular membership function used in this example; 
the simulation results for the robust fuzzy control are shown in figure 20. The tracking error time histories 
for each of the joint angles are shown, indicating that the fuzzy controller provides good tracking 
performance. It even provides slightly better performance than a pure nonlinear robust controller does. 



Figure 19. Symmetric triangular membership functions. 
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Figure 20. Position error time histories. 


3.3.2 . Barriers 

The controls discipline is afflicted by barriers B1-B3, B6 and B7. In the controls context B6 would be 
phrased in terms of unmodeled dynamics. In a sense the controls counterpart of B7 presents an even 
greater challenge because the low-dimensional aircraft models used in controls represent a much more 
drastic simplification of the detailed physics than the models of concern in aerodynamics. 

The following are some barriers that are specific to applying probabilistic methods to the design and 
analysis of closed-loop dynamical systems: 

B8. Characterization of uncertainties for use in control is inadequate. 

Control design obviously requires the characterization of parameter and model form uncertainties from 
a broad range of disciplines. Clearly, the uncertainty characterization activities in such disciplines as 
structures and aerodynamics need to take into account the requirements flowing from the controls 
community. 

B9. Methods for mapping probabilistic parameter uncertainties into norm-bounded uncertainties do not 
exist. 

Conventional robust control approaches rely on norm-bounded descriptions for uncertainties in both 
analysis and design. An enormous amount of work has been done in this area and it is highly desirable to 
leverage this work by finding ways to incorporate probabilistic uncertainties into the robust control design 
and analysis processes. This barrier implies that insertion of probabilistic information into controls 
analysis and design processes requires a substantial overhaul of these processes. 

BIO. Existing probabilistic analysis tools are not well suited to handle the time and frequency domain 
response quantities that are typically used in the analysis of closed-loop dynamical systems. 

Existing FORM and SORM methods are geared towards steady problems. Clearly, similarly effective 
methods are needed for unsteady problems with the high modal densities (zeros and poles) that are 
common in the control of dynamical systems. 





3.4. Optimization 

3.4.1. Current Status 

Multidisciplinary Design Optimization (MDO) is increasingly important to the aerospace community. 
For example. Giesing and Barthelemy (1998) summarize presentations at the 7th AIAA Multidisciplinary 
Analysis and Optimization Symposium addressing the uses of MDO in industry. The industry 
representatives provided many examples of successful MDO applications but also produced a list of new 
methods that the industry requires. One of those emerging areas was robust design or optimization 
methods that produce solutions insensitive to variability in input parameters. 

The need for robust design methods appears in many contexts. During the preliminary design process, 
the exact value of input parameters is not known. It may be possible to make an educated guess or provide 
bounds for these unknown parameters but they are not deterministic quantities. Faced with uncertain 
parameters, traditional optimization techniques tend to “over-optimize”. Like the curve labeled “Sharp" in 
figure 21. solutions produced by these techniques perform well at the design point but have poor off- 
design characteristics. As noted by Young, Anderson, and Yurkovich (1998), the aerospace industry 
favors designs that have room to grow. Like the curve labeled “Robust” in figure 21. designs required by 
the industry must be adaptable to new missions or new business climates without a marked decrease in 
performance and is often willing to sacrifice a sharp optimum for this flexibility. 


Performance 



Figure 21 . Illustration of robust optimum. 

The present research seeks optimization methods that are robust in the sense that they produce 
solutions insensitive to small changes in the input parameters. As an additional requirement, the methods 
must be able to find successful designs using a moderate number of high-fidelity disciplinary analyses. 
This second requirement acknowledges the fact that disciplinary analyses (e.g. CFD) can be 
computationally expensive and an optimization method that requires thousands of function evaluations 
has limited usefulness in the current design environment. 

An equally important research area is optimization for reliability, which is not a new idea. Some of the 
earliest examples of structural optimization include reliability constraints. Today, the open issues in 
reliability-based structural optimization involve testing and validating the optimization procedures and 
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demonstrating the benefits of reliability-based methods over conventional methods. Other interesting 
areas for study involve a comparison of possibilistic methods with probabilistic methods. Chen et al. 
(1999) conclude that possibilistic methods are preferred when there is insufficient data to build accurate 
probabilistic models of the uncertainties. 

3.4. 1.1. Sampling-Based Methods. Response surface methods and other sampling-based methods are 
widely admired and often used but are seldom systematically applied. For example, Taguchi parameter 
design methods are popular because they prescribe both experimental test matrices and the objective 
function and they provide detailed instructions about how to determine the best design. The popularity of 
Taguchi methods has encouraged several researchers to use them as the basis for optimization w ith 
uncertainty. For example, Lee et al. (1996) describe their use for solving unconstrained engineering 
problems, and Unal, Stanley, and Joyner (1993) used Taguchi methods as a tool for conceptual design of 
propulsion systems. Like Taguchi methods, Latin hypercubes and other random sampling methods have 
been used for optimization with uncertainty. For example, DeLaurentis and Mavris (2000) demonstrate 
robust design on a high-speed civil transport and Booker et al. (1998) apply pattern search methods to 
optimize rotor blades. These methods all prescribe data points forevaluation and approximate responses 
from simulations. 

These methods also share a common weakness by ignoring bias errors in simulation or experimental 
results. Booker et al. (1998) emphasize that sampling-based optimization methods are useful if the 
simulation produces anomalous results for some combinations of input. Torczon and Trosset (1998) 
explain that pattern search optimization methods rarely converge to local minima that are caused by 
truncation, rounding or other numerical errors in simulation codes. Thus, these authors recommend their 
methods for optimization with uncertainty. However, none of the authors acknowledge that bias errors in 
the predicted responses can invalidate the optimization results. For example, the Taguchi method ignores 
the possibility that variations in data collected by different facilities or by the same facility in different 
circumstances may not be due solely to random noise. Similarly, data sampling methods ignore the 
possibility that simulations include bias errors due to a different fidelity of analysis or different grid 
generation techniques. Applying sampling to a single source of data avoids some of the bias error issues 
but can produce useless optimization results as the optimization procedure exploits weaknesses in the 
approximate model. On the other hand, applying optimization to data from a mixture of experimental and 
mathematical simulation sources requires an ad hoc procedure. Romero (1999) discusses the bias error 
problem and suggests a global-local approach that works well on some demonstration problems. Patera 
(1997) discusses errors in experimental and simulated data and devises a method to estimate the resulting 
error in the optimization results. 

The most promising work in this area is coordinated by Sandia National Laboratories ASCI 
Verification and Validation program (Alvin et al. 2000). The first step is to identify sources of variability , 
uncertainty, and error in simulations. The next step is to develop both approximation and sampling 
methods that properly account for this uncertainty. Once uncertainty is characterized, then currently 
available optimization procedures can be applied. Romero et al. ( 1995) demonstrate the power and 
potential of these methods by using optimization to identify worst-case thermal loading conditions. They 
conclude that a warehouse fire might compromise the fail-safe mechanisms on military stores unless the 
proposed design of the firing system is modified to account for uncertainty. A similar technique could be 
effective in aerospace design, for example, by identifying worst-case aeroelastic loads or by discovering 
wind tunnel test matrices that exaggerate the difference between turbulence models. 

3.4. 1:2. Robust Optimization Methods. Robust optimization methods similar to equation (2) have been 
proposed for many years but have seldom been evaluated for practical applications. Yoon, Jung, and 
Hyun (1999) propose robust optimization as a way to specify machine tolerances for simple structural 
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sizing problems. And Teng, Free, and Parkinson (1992) demonstrate the value of these methods for 
simplified active and passive vibration control systems. Similarly, Darlington et al. (1999) consider batch 
chemical reactors and use robust optimization to design the thermal control system. Du and Chen (2000a, 
2000b) extend these ideas to MDO problems. They argue that most MDO procedures assume that each 
disciplinary analysis predicts with equal certainty and that very different solutions will result if these 
problems are posed using robust optimization. 

Robust optimization methods have two major weaknesses: the difficulty in choosing weights and the 
lack of confidence in the estimate of variance. Messac and Sundararaj (2000) address the difficulty in 
choosing weights. They explain that multiobjective formulations that depend on a weighted sum of 
objective functions are designed to find a compromise that reduces each objective. However, this 
weighted sum method often fails because no set of weights allows the optimization procedure to find the 
best compromise. Moreover, even if a good choice of weights does exist, the user has no way to know 
those weights. Messac and Sundararaj (2000) demonstrate an alternative approach called Physical 
Programming. Frangopol and Iizuka (1992) suggest another approach called e-constraint method and 
show how this method can be used in structural designs with many failure modes. 

Even if a superior multiobjective formulation is discovered, the lack of confidence in cf will remain a 
weakness of these methods. Current methods such as Monte Carlo analysis are computationally expensive 
and often require human intervention in order to operate reliably. Some researchers seek better ways to 
predict cr . For example, Taylor (2000) suggests using automatic differentiation techniques to predict first 
and second derivatives of c with respect to 6. Taylor compares the resulting estimates of variance using 
this new method and using Monte Carlo analysis. Similarly, Archetti, Gaivoronski, and Stella (1997) 
consider efficient gradient estimation procedures and their impact on robust optimization. 

Several researchers are exploring new avenues in robust optimization. For example, Huyse and Lewis 
(2001) suggest a formulation 


min \ c(t,6)fJd)dd (7) 

ten e 

where /i/0) is the PDF that describes the uncertain parameters 0. Chosen design variables should 
minimize the expected value of the objective function. As an example, the shape of an airfoil profile and 
the angle of attack needed to provide required lift are optimized for free stream Mach numbers ranging 
from 0.7 to 0.8. The Mach number range can be a uniform distribution if all Mach numbers are equally 
important or it can be a Beta distribution centered on the design cruise Mach number. This new 
formulation is exciting because it avoids the over-design phenomena seen in single point designs, yet it 
requires no more function evaluations than the multipoint designs currently used by the airframe industry. 

3.4.I.3. Optimization for Reliability. Optimization for reliability is the most studied and used of the 
methods for optimization with uncertainty. Civil engineering and aerospace structural analysts are highly 
motivated to produce designs with a small probability of failure. Simulations that predict structural 
response as well as reliability are becoming commercially available. Moreover, new materials such as 
composite polymers and metal matrices require new design tools that take advantage of their special 
properties without compromising safety (Abumeri, Kuguoglu, and Chamis 2000). 

Gas turbine design and other aerospace propulsion systems design have benefited the most from 
current research. Abumeri and Chamis (2000) provide an overview of an extensive capability for 
optimization of engine structures. Grandhi and Wang (1999) show the benefit of designing turbine blades 
for reduced weight and increased reliability. Kowal and Mahadevan (1998) demonstrate the surprising 
conclusion that extremely tight tolerances in manufacturing turbine blades can be counterproductive; they 
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use manufacturing tolerances as design variables and conclude that a certain amount of irregularity 
actually improves the reliability of turbines. 

In addition to optimization tools, the aerospace community has also provided validation cases, which 
increase the credibility of these methods. For example, Boeing engineers report cost savings due to 
reliability studies related to the redesign of the Shuttle docking bay for use in Shuttle-Mir rendezvous 
(Torng and Yang 1994; Torng, Funk, and Stephenson 1999). 

3.4.2. Barriers 

The one glaring feature of the survey of methods for optimization under uncertainty is that all existing 
methods are designed for probabilistic descriptions of uncertainty. Few researchers work on optimization 
under uncertainties characterized by other descriptions, such as by intervals or by membership functions. 
Yet Chen (1999) suggests that probabilistic methods are only effective if sufficient data exists to build 
accurate probabilistic models. Hence, 

B1 1. No methods are available for optimization under nonprobabilistic uncertainties. 

This barrier is especially formidable because plausible probabilistic descriptions are never likely to be 
available for most effects due to uncertainties and errors. 

Even within the realm of probabilistic characterizations of uncertainty, barrier B3 is even more 
formidable for optimization under uncertainty than for the individual disciplinary uncertainty analyses. It 
is worth paraphrasing it here with special emphasis on the optimization technology challenge: 

B12. Current methods for optimization under probabilistic uncertainty are too expensive for use with 
high-fidelity analysis tools in many disciplines. 

Clearly, a breakthrough in algorithms for optimization under uncertainty would have wide ranging 
impact because it would ameliorate the barrier of computational expense for uncertainty analysis in many 
disciplines. 

B 1 3. Extending uncertainty analysis and optimization to applications involving multiple disciplines 
compounds the complexity and cost. 

Most (perhaps all) current approaches to multidisciplinary uncertainty analysis and optimization treat 
the entire multidisciplinary analysis as a unit — no attempts have been made to apply decomposition 
techniques to improve the efficiency. In the realm of deterministic problems, MDO methods have been 
developed that decompose the full multidisciplinary problem into more manageable components, usually 
along disciplinary lines. These decomposition approaches have produced more computationally efficient 
schemes than one gets by simply slapping a generic optimizer on top of a fully integrated 
multidisciplinary analysis code. (No comprehensive survey of these approaches exists. See Alexandrov 
2001 for a review of multilevel decomposition approaches and Alexandrov and Lewis 2000 for some 
work on distributed optimization.) Hopefully similarly creative approaches to problem decomposition can 
mitigate this barrier, both for uncertainty analysis and for optimization under uncertainty. 


3.5. Summary of Barriers 

The preceding sections have mentioned a number of barriers to the adoption of uncertainty-based 
design methods for aerospace vehicles. In summary: 
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Bl. Industry feels comfortable with traditional design methods 

B2. Few demonstrations of the benefits of uncertainty-based design methods are available. 

B3. Current uncertainty-based design methods are more complex and much more 
computationally expensive than deterministic methods. 

B4. Characterization of structural imperfections and uncertainties necessary to facilitate accurate 
analysis and design of the structure is time consuming and is highly dependent on structural 
configuration, material system, and manufacturing processes. 

B5. There is a dearth of statistical process control activity in aerodynamics. 

B6. Effective approaches for characterizing model form error are lacking. 

B7. There are no dependable approaches to uncertainty quantification for nonlinear problems. 

B8. Characterization of uncertainties for use in control is inadequate. 

B9. Methods for mapping probabilistic parameter uncertainties into norm-bounded uncertainties 
do not exist. 

BIO. Existing probabilistic analysis tools are not well suited to handle the time and frequency 
domain response quantities that are typically used in the analysis of closed-loop dynamical 
systems. 

Bl 1. No methods are available for optimization under nonprobabilistic uncertainties. 

B 1 2. Current methods for optimization under uncertainty are too expensive for use with high- 
fidelity analysis tools in many disciplines. 

B 1 3. Extending uncertainty analysis and optimization to applications involving multiple 
disciplines compounds the complexity and cost. 

B14. Researchers and analysts lack training in statistical methods and probabilistic assessment. 

This last barrier has not been mentioned previously. We list it here because it is a very real, pervasive 
barrier that must be tackled. The developers of uncertainty-based design methods must substantially 
increase their knowledge of these subjects. Moreover, the intended end users of uncertainty-based design 
methods — the analysts — must become more acquainted with this area. 


4. Potential Benefits of Uncertainty-Based Design 

Previous sections have mentioned a number of potential benefits of uncertainty-based design, most of 
which apply to all the disciplines as well as to the multidisciplinary system (though not necessarily to the 
same degree). 

PI. Confidence in analysis tools will increase. 

Uncertainty-based design methods are arguably the enabling technology for turning the use of analysis 
tools from an art into a science. To accomplish this transformation, a comprehensive strategy for 
assessing uncertainty must be developed. The focus today within most computational disciplines is on 
quantifying and managing discretization and convergence error and on calibrating codes against 
experimental data. This narrow view must be greatly expanded in order to provide the complete picture of 
uncertainty that is essential for inspiring confidence in analysts and project managers. As one disciplinary 
example, CFD is a critical design tool which will have increased use if variations can be quantified and 
contained, together with quantifying and containing the variation of experimental results used for code 
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and solution (conceptual/preliminary design) validation. 

P2. Design cycle time, cost, and risk will be reduced. 

One key to reducing the design cycle time is to avoid computational overkill. Knowing the uncertainty 
associated with codes of various fidelities would permit identification of the fastest codes that met the 
uncertainty requirements of the project. In other words, uncertainty-based design methods would enable 
quantifiable trade-offs between computational uncertainty and computational expense. Moreover, upfront 
knowledge of where the uncertainties in the design tools are greatest can lead to more efficient use of 
risk-reduction experiments. 

P3. System performance will increase w'hile ensuring that reliability requirements are met. 

Aerospace vehicles are as safe as they are today in large part because of the conservative approach to 
their design. Uncertainty-based design methods provide a means to increase system performance while 
ensuring that the reliability or safety requirements are still met. Certainly other fields of engineering have 
demonstrated that structural weight can be reduced without sacrificing safety requirements. Also, greater 
performance can potentially be obtained from MIMO control systems by replacing the norm-bounded 
approach to uncertainties (structured singular values) in control system design and analysis with genuine 
probabilistic methods. 

P4. Designs will be more robust. 

Uncertainty-based optimization methods can identify good designs that minimize the impact of 
uncertainties arising from the system operating conditions, manufacturing variabilities, and the design 
tools themselves. 

P5. The methodology can assess systems at off-nominal conditions. 

Uncertainty-based design methods provide a systematic means to address and analyze systems or 
subsystems that may operate in abnormal, unusual, or damaged conditions. Such approaches can be used 
to address the degraded performance or failure of such systems. 

P6. The use of composite structures will increase. 

This benefit is the one discipline-specific benefit that we call out in this list. The payoff for application 
of probabilistic design is greater for composite structures than for metallic structures because of the 
greater variability of the materials used in composites. 


5. Proposed LaRC Uncertainty-Based Design Research 

The proposed role for NASA Langley Research Center in uncertainty-based design is: 

Evaluate and improve methods for management of uncertainty with applications to 
multidisciplinary aerospace vehicle design by developing and validating strategies, algorithms, 
tools and data to 

characterize and manage the uncertainties from the individual aerospace vehicle design 
disciplines, especially aerodynamics, structures, and controls, based on the best available 
experimental and computational results; 

characterize the norm and distribution of the resulting uncertainties in system metrics; 
and 
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account for uncertainties in the design of aerospace vehicles at the conceptual through the 
detailed design stages. 

The key phrases for the NASA LaRC role are “multidisciplinary,” “aerospace vehicle design 
disciplines,” “experimental and computational," “system metrics,” and “at the conceptual through the 
detailed design stages.” Certainly some aspects of uncertainty management require sophisticated 
information technology; the role of LaRC is to exploit mature information technologies as applied to the 
airframe design disciplines but not necessarily to perform fundamental research on new information 
technologies. 

Obviously, research is much needed in these broad areas. The essential needs are presented in tables in 
the next three subsections. Our assessment of the current Technology Readiness Level (TRL) is given in 
the third column of each table. The definitions of the TRLs can be found in appendix A. The items 
selected for emphasis in a LaRC uncertainty-based design program are chosen for their potential to 
overcome the various barrier issues described in section 3 and because their TRLs must be raised before 
they become candidates for application to focused programs. 


5.1. Characterizing and Managing Disciplinary Uncertainties 

Uncertainty characterization begins at the individual discipline level. Uncertainty management can 
only be effected at the discipline level, although it can be directed (or budgeted) at the system level. The 
general needs for the structures, aerodynamic and controls disciplines are to given in tables 2, 3, and 4. 


Table 2. Characterizing and Managing Structural Uncertainties 


Goal 

Description 

Current 

TRL 

SI 

Develop fundamental experimental testing methods to characterize 
structural responses with an emphasis on managing uncertainties 

2 

S2 

Develop and validate fundamental algorithmic enhancements to high- 
fidelity structural analysis codes and to implement these codes in an 
uncertainty-based design framework 

3 

S3 

Develop and validate modeling error estimation capabilities in structural 
analysis codes 

2 

S4 

Characterize the structural response of aircraft for use in reliability-based 
structural design, including aerodynamic and nonaerodynamic loads 

2 

S5 

Characterize the structural parameter uncertainties of aircraft for use in 
robust structures and control design 

2 
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Table 3. Characterizing and Managing Aerodynamics Uncertainties 


Goal 

Description 

Current 

TRL 

A1 

Refine and extend the use of statistical process control and modem design 
of experiments strategies in experimental aerodynamics for data from 
individual facilities 

4 

A2 

Develop new statistical process control strategies for data from multiple 
facilities 

3 

A3 

Develop statistical process control strategies for computational 
aerodynamics 

3 


Develop efficient global sensitivity analysis algorithms and second-order 
local sensitivity analysis algorithms to characterize parameter 
uncertainties from nonlinear aerodynamics tools 

3 

A5 

Develop and validate strategies to characterize the model form 
uncertainties associated with transition and turbulence, vortical flows, 
separation from smooth surfaces, jets, wakes, reattaching flows, re- 
laminarizing flows, shock-boundary-layer interaction, unsteadiness, etc. 

2 

A6 

Develop strategies to characterize the aerodynamic loads on aircraft for 
use in structural reliability-based design 

2 

A7 

Develop strategies to characterize the aerodynamic uncertainties on 
aircraft for use in robust control design 

2 


Table 4. Characterizing Controls Uncertainties 


Goal 

Description 

Current 

TRL 

Cl 

Characterize the uncertainties in sensors and actuators, e.g., sensor and 
actuator noise drift, and bias 

2 

C2 

Characterize uncertainties in the control design models obtained by using 
system identification techniques 

2 
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5.2. Characterizing Uncertainties in System Metrics 

The general needs in multidisciplinary analysis are presented in table 5 . 


Table 5. Characterizing System Uncertainties 


Goal 

Description 

Current 

TRL 

VI 

Characterize the uncertainties in the controls performance measures 

2 

V2 

Develop strategies to obtain efficient uncertainty propagation 
characterizations using disciplinary codes equipped with efficient 
sensitivity derivatives 

3 

V3 

Develop strategies to produce system uncertainty characterizations based 
on a combination of computational and experimental uncertainty 
characterizations 



53. Accounting for Uncertainties in Airframe Design 

The general needs for airframe design are presented in table 6. 


Table 6. Accounting for Uncertainty in Airframe Design 


Goal 

Description 

Current 

TRL 

Ml 

Partner with industry and the FAA in the application and validation of 
probabilistic reliability-based design methods for composite airframe 
structures 

2 

M2 

Develop viable uncertainty-based design methods that span the design 
cycle from conceptual through detailed design 

2 

M3 

Develop and validate robust aerodynamic design methods for 
aerodynamic performance optimization 

3 

M4 

Develop and validate reliability-based aerodynamic design methods for 
control effectiveness 

1 

M5 

Develop and validate probabilistic-based algorithms and tools for robust 
control design and analysis of aerospace systems 

2 

M6 

Develop and validate probabilistic-based algorithms and tools for 
reliability-based optimization and robust optimization 

2 

M7 

Develop strategies to characterize confidence in optimal solutions 

2 

M8 

Educate the engineering community at LaRC on nondeterministic 
problem formulations of their engineering problems, the potential benefits 
of nondeterministic approaches, and the exciting research opportunities in 
this field 

NA 
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5.4. Approach 

The approach to meet the listed needs will depend on the maturity of uncertainty-based design 
technology in the various disciplines. Early studies may be exploratory, pathfinding studies that explain 
procedures and indicate possible benefits of RBD. These early studies may not involve complex 
mathematical models. Later studies will involve realistic mathematical models and high-fidelity analyses 
for single disciplines — then multiple disciplines. These later studies will include strategies for 
incorporating high-performance computing. 

6. Conclusions 

6.1. Expected Results 

The expected concrete results from a 5-year uncertainty-based design research activity are shown in 
table 7. In addition to providing a short goal statement with the expected output and outcome, we also 
furnish our estimate of the Research and Development Degree of Difficulty (R&D) using the scale 
developed by Mankins (1998): 

• R&D 3 - Level I: A very low' degree of difficulty is anticipated in achieving R&D objectives for this 
technology. Probability of success in normal R&D effort — 99 percent 

• R&D 3 - Level II: A moderate degree of difficulty should be anticipated in achieving R&D 
objectives for this technology. Probability of success in normal R&D effort — 90 percent 

• R&D 3 - Level III: A high degree of difficulty anticipated in achieving R&D objectives for this 
technology. Probability of success in normal R&D effort — 80 percent 

• R&D 3 - Level IV: A very high degree of difficulty anticipated in achieving R&D objectives for this 
technology. Probability of success in normal R&D effort — 50 percent 

3 

• R&D - Level V: The degree of difficulty anticipated in achieving R&D objectives for this 
technology is so high that a fundamental breakthrough is required. Probability of success in normal 
R&D effort — 20 percent 

An expected intangible outcome of the concrete results of this activity will be the conversion of dozens of 
other LaRC engineers to nondeterministic approaches to airframe design. 

We also expect substantial interim benefits such as 

1 . New, inexpensive strategies for quantifying and controlling computational and experimental 
uncertainties in the disciplines. 

2. New, less-expensive strategies for verifying and validating computational codes and solutions. 

3. Significant insight into the process of Measuring Predictive Capability. 

a. What is it? 

b. How do we carry it out in the face of severe nonlinearities and inference domains 
separated by different physics? 
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6.2. Opportunities for LaRC 

LaRC has a window of opportunity to be in the vanguard of uncertainty-based methods for 
multidisciplinary airframe design. Momentum is gathering in the engineering community for this 
technological leap. LaRC has tremendous advantages in this nascent field, and can facilitate the adoption 
of uncertainty-based methods by industry and NASA programs. 


Table 7. Expected Results from Uncertainty-Based Design Research 


Goal 

Output 

Outcome 

R&D 3 

Level 

Affordable 

probabilistic 

analyses 

Augment high-fidelity structures 
and aerodynamics tools with 
efficient probabilistic output 

Codes provide estimates of mean and 
standard deviation and/or probability 
of failure 

ii 

Possibilistic 

uncertainty 

quantification 

Develop fundamental possibilistic 
strategies and algorithms for 
multidisciplinary uncertainty 
quantification 

Reduces system risk by enabling 
uncertainty quantification and design 
for uncertainty strategies at the 
conceptual design stage 

IV 

Aerodynamics 

uncertainty 

quantification 

Develop efficient uncertainty 
quantification and control 
strategies and algorithms for 
aerodynamics performance, loads, 
and stability and control 
predictions 

Increases design confidence and 
aircraft safety by measuring and 
controlling uncertainty in 
aerodynamics performance predictions 
and by enabling reliability-based 
structural design, stochastic control 
design, and reliability-based 
aerodynamic controllability analysis 

III 

Robust 

aerodynamic 

optimization 

Develop 3-D, aerodynamic shape 
optimization algorithms that 
provide designs which are robust 
with respect to uncertainties in 
geometry, operating conditions, 
and CFD code accuracy 

Controls and reduces risk by providing 
designs with aerodynamic performance 
that is insensitive to intrinsically 
uncertain quantities 

III 

Reliability-based 
structural design 

Extend CSM and experimentally 
! verify to predict probability of 
failure for composite structures 

Airframe components designed with 
consistent levels of reliability 

IV 

Probabilistic 
controls analysis 
and design 

Develop MIMO designs that use 
probabilistic input data 

Safer and less conservative active 
control of aircraft 

III 

Multidisciplinary 

uncertainty-based 

design 

Demonstrate multidisciplinary 
uncertainty-based airframe design 
incorporating the aerodynamics, 
structures and controls disciplines 

Increases design confidence, controls 
and reduces design risk, and increases 
aircraft safety by enabling 
interdisciplinary design trade-offs that 
explicitly account for uncertainty, risk, 
and safety 

V 
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Appendix A. 


Technology Readiness Levels 

The Technology Readiness Levels (TRLs) are defined as follows: 

TRL 9: Actual system “mission proven” through successful mission operations. Thoroughly 
debugged software readily repeatable. Fully integrated with operational hardware/software systems. All 
documentation completed. Successful operational experience. Sustaining software engineering support in 
place. Actual system fully demonstrated. 

TRL 8: Actual system completed and “mission qualified” through test and demonstration 
in an operational environment. Thoroughly debugged software. Fully integrated with operational 
hardware and software systems. Most user documentation, training documentation, and maintenance 
documentation completed. All functionality tested in simulated and operational scenarios. Verification 
and validation completed. 

TRL 7: System prototype demonstration in high-fidelity environment (parallel or shadow 
mode operation). Most functionality available for demonstration and test. Well integrated with 
operational hardware/software systems. Most software bugs removed. Limited documentation available. 

TRL 6: System/subsystem prototype demonstration in a relevant end-to-end environment. 

Prototype implementations on full-scale realistic problems. Partially integrated with existing 
hardware/software systems. Limited documentation available. Engineering feasibility fully demonstrated. 

TRL 5: Module and/or subsystem validation in relevant environment. Prototype 
implementations conform to target environment/interfaces. Experiments with realistic problems. 

Simulated interfaces to existing systems. 

TRL 4: Module and/or subsystem validation in laboratory environment. Stand-alone 
prototype implementations. Experiments with full-scale problems or data sets. 

TRL 3: Analytical and experimental critical function and/or characteristic proof-of- 
concept. Limited functionality implementations. Experiments with small representative data sets. 
Scientific feasibility fully demonstrated. 

TRL 2: Technology concept and/or application formulated. Basic principles coded. Experiments 
with synthetic data. Mostly applied research. 

TRL 1: Basic principles observed and reported. Basic properties of algorithms, representations, 
and concepts. Mathematical formulations. Mix of basic and applied research. 


53 



REPORT DOCUMENTATION PAGE 


Form Approved 
OMB No. 0704-0188 


The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources 
gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 
collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 
Reports (0704-0188). 1215 Jefferson Davis Highway, Suite 1204, Arlington. VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law no person 
shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number 
PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 


2. REPORT TYPE 

Technical Memorandum 


1. REPORT DATE (DD-MM-YYYY) 
07-2002 


4. TITLE AND SUBTITLE 

Needs and Opportunities for Uncertainty-Based Multidisciplinary Design 
Methods for Aerospace Vehicles 


3. DATES COVERED (From - To) 


5a. CONTRACT NUMBER 


5b. GRANT NUMBER 


5c. PROGRAM ELEMENT NUMBER 


5d. PROJECT NUMBER 


6. AUTHOR(S) 

Thomas A. Zang, Michael J. Hemsch, Mark W. Hilburger, Sean P. Kenny, 

James M. Luckring, Peiman Maghami, Sharon L. Padula, and W. Jefferson 5 e task NUMBER 
Stroud 

5f. WORK UNIT NUMBER 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

NASA Langley Research Center 
Hampton, V A 23681-2199 


1706-31-61-01 


8. PERFORMING ORGANIZATION 
REPORT NUMBER 


L- 1 8174 


9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 

National Aeronautics and Space Administration 
Washington, DC 20546-0001 


10. SPONSOR/MONITOR'S ACRONYM(S) 

NASA 


11. SPONSOR/MONITOR'S REPORT 
NUMBER(S) 

NASA/TM-2002-2 1 1462 


12. DISTRIBUTION/AVAILABILITY STATEMENT 

Unclassified - Unlimited 
Subject Category 05 

Availability: NASA CASI (301)621-0390 Distribution: Standard 


13. SUPPLEMENTARY NOTES 

An electronic version can be found at http://techreports.larc.nasa.gov/ltrs/ or http://techreports.larc.nasa.gov/cgi-bin/NTRS 


14. ABSTRACT 


This report consists ot a survey of the state of the art in uncertainty-based design together with recommendations for a Base 
research activity in this area lor the NASA Langley Research Center. This report identifies the needs and opportunities for 
computational and experimental methods that provide accurate, efficient solutions to nondeterministic multidisciplinary 
aerospace vehicle design problems. Barriers to the adoption of uncertainty-based design methods are identified, and the benefits 
of the use of such methods are explained. Particular research needs are listed. 


15. SUBJECT TERMS 

Aeronautics, Applied Mathematics, Systems Analysis, Uncertainty Quantification 


16. SECURITY CLASSIFICATION OF: 

a. REPORT 

b. ABSTRACT 

c. THIS PAGE 

U 

u 

u 


17. LIMITATION OF I 18. NUMBER! 19a. NAME OF RESPONSIBLE PERSON 


ABSTRACT 


OF 

PAGES 


STI Help Desk (email: help<s>sti. nasa.gov) 


19b. TELEPHONE NUMBER (Include area code) 

(301)621-0390 


Standard Form 298 (Rev. 8-98) 

Prescribed by ANSI Sid Z39.18 























